{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"![](\"https://i.imgur.com/ZeJluqa.png\")
\n\nGreat Barrier Reef - YOLO Full Understanding
\n\n# Introduction\n\n> š **Note1**: Thank you to [Awsaf](https://www.kaggle.com/awsaf49) and his amazing notebook **[Great-Barrier-Reef: YOLOv5 train š](https://www.kaggle.com/awsaf49/great-barrier-reef-yolov5-train)**, which introduced me and got me started onto this amazing Object Detection path. š I could not have done it without him, as this is my first time getting acquinted to YOLO (what is it, how to use it, how to code it). His 2 notebooks (the one already mentioned and **[Great-Barrier-Reef: YOLOv5 [infer] š](https://www.kaggle.com/awsaf49/great-barrier-reef-yolov5-infer)**) are amazing starters onto this world.\n\n> š **Note2**: Here is the [official YOLOv5 user notebook on Kaggle by Ultralytics](https://www.kaggle.com/ultralytics/yolov5). It contains a full tutorial on how to get started. \n\n## What is YOLO?\n\nš **YOLO**: You Only Look Once (contrary to my initial beliefs of thinking it was related to You-Only-Live-Once š)\n\nš **What is it?** YOLO it's a very simple and fast algorithm that recognizes objects within an image in real time. It is made up by a single CNN and requires only one forward pass through the neural network in order to identify the objects.\n\nš **How does it work?**\n\n1. The image is split in a grid that has the same dimension for each \"tile\".\n2. We add the bounding boxes that identify each object. The bbox has the following format: `[width, height, class, bx, by]`, where `[bx, by]` represents the center of the object.\n3. Intersection Over Union: this technique is used so the bounding box \"catches\" the object fully (and doesn't leave any part of it uncovered, neither it is too large for the object). The `IOU=1` if the predicted and actual box are identical.\n\n![](\"https://i.imgur.com/Ce1sfqj.png\")
\n\n*Source: [here](https://www.section.io/engineering-education/introduction-to-yolo-algorithm-for-object-detection/)*\n\n> **Disclaimer**: as this is my very first attempt at an Object detection task, I might be wrong in some cases. If you observe anything odd, please do address it in the comments ^^.\n\n### ā¬ Libraries Below","metadata":{}},{"cell_type":"code","source":"# Libraries\nimport os\nimport sys\nimport wandb\nimport torch\nimport time\nimport random\nimport shutil\nimport yaml\nfrom tqdm import tqdm\nimport warnings\nimport cv2\nimport pandas as pd\nimport numpy as np\nimport seaborn as sns\nimport matplotlib as mpl\nimport matplotlib.patches as patches\nimport matplotlib.pyplot as plt\nfrom IPython.display import display_html\n\n\n# Environment check\nwarnings.filterwarnings(\"ignore\")\nos.environ[\"WANDB_SILENT\"] = \"true\"\nCONFIG = {'competition': 'greatReef', '_wandb_kernel': 'aot'}\n\n# š Secrets\nfrom kaggle_secrets import UserSecretsClient\nuser_secrets = UserSecretsClient()\nsecret_value_0 = user_secrets.get_secret(\"wandb\")\n\n! wandb login $secret_value_0\n\n# Custom colors\nclass color:\n S = '\\033[1m' + '\\033[94m'\n E = '\\033[0m'\n \nmy_colors = [\"#16558F\", \"#1583D2\", \"#61B0B7\", \"#ADDEFF\", \"#A99AEA\", \"#7158B7\"]\nprint(color.S+\"Current Directory\"+color.E, os.getcwd())\nprint(color.S+\"Notebook Color Scheme:\"+color.E)\nsns.palplot(sns.color_palette(my_colors))","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:07:10.331535Z","iopub.execute_input":"2022-02-05T10:07:10.332606Z","iopub.status.idle":"2022-02-05T10:07:17.172869Z","shell.execute_reply.started":"2022-02-05T10:07:10.332462Z","shell.execute_reply":"2022-02-05T10:07:17.170955Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### ā¬ Helper Functions Below","metadata":{}},{"cell_type":"code","source":"# === š W&B ===\ndef save_dataset_artifact(run_name, artifact_name, path):\n '''Saves dataset to W&B Artifactory.\n run_name: name of the experiment\n artifact_name: under what name should the dataset be stored\n path: path to the dataset'''\n \n run = wandb.init(project='g2net', \n name=run_name, \n config=CONFIG, anonymous=\"allow\")\n artifact = wandb.Artifact(name=artifact_name, \n type='dataset')\n artifact.add_file(path)\n\n wandb.log_artifact(artifact)\n wandb.finish()\n print(\"Artifact has been saved successfully.\")\n \n \ndef create_wandb_plot(x_data=None, y_data=None, x_name=None, y_name=None, title=None, log=None, plot=\"line\"):\n '''Create and save lineplot/barplot in W&B Environment.\n x_data & y_data: Pandas Series containing x & y data\n x_name & y_name: strings containing axis names\n title: title of the graph\n log: string containing name of log'''\n \n data = [[label, val] for (label, val) in zip(x_data, y_data)]\n table = wandb.Table(data=data, columns = [x_name, y_name])\n \n if plot == \"line\":\n wandb.log({log : wandb.plot.line(table, x_name, y_name, title=title)})\n elif plot == \"bar\":\n wandb.log({log : wandb.plot.bar(table, x_name, y_name, title=title)})\n elif plot == \"scatter\":\n wandb.log({log : wandb.plot.scatter(table, x_name, y_name, title=title)})\n \n \ndef create_wandb_hist(x_data=None, x_name=None, title=None, log=None):\n '''Create and save histogram in W&B Environment.\n x_data: Pandas Series containing x values\n x_name: strings containing axis name\n title: title of the graph\n log: string containing name of log'''\n \n data = [[x] for x in x_data]\n table = wandb.Table(data=data, columns=[x_name])\n wandb.log({log : wandb.plot.histogram(table, x_name, title=title)})","metadata":{"_kg_hide-input":true,"jupyter":{"source_hidden":true},"execution":{"iopub.status.busy":"2022-02-05T10:07:17.176058Z","iopub.execute_input":"2022-02-05T10:07:17.176429Z","iopub.status.idle":"2022-02-05T10:07:17.190625Z","shell.execute_reply.started":"2022-02-05T10:07:17.176382Z","shell.execute_reply":"2022-02-05T10:07:17.189598Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Step 1. Data Prep\n\nWhen using YOLOv5 we will have to work more with folders and basic *command lines*, vs the usual coding techniques that are encountered in other competitions.\n\nBy default, we are within the `/kaggle/working` directory. Whenever you want to `save` a file, the output goes here.\n\nOther 2 directories within the `/kaggle/` directory are `lib` and `input`, where you can find the datasets we are using.\n\nš The idea here is to create 2 more files: \n* `/kaggle/images` - empty folder where we will store our training images\n* `/kaggle/labels` - empty folder where we will store out labels (or annotations) found within these images","metadata":{}},{"cell_type":"code","source":"print(color.S+\"-Directory Structure-\"+color.E)\nprint(color.S+\"Before:\"+color.E, os.listdir(\"../\"))\n\n# Create 2 new folders\n!mkdir -p '../images'\n!mkdir -p '../labels'\n\nprint(color.S+\"After:\"+color.E, os.listdir(\"../\"))","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:07:17.192687Z","iopub.execute_input":"2022-02-05T10:07:17.193896Z","iopub.status.idle":"2022-02-05T10:07:18.721742Z","shell.execute_reply.started":"2022-02-05T10:07:17.193833Z","shell.execute_reply":"2022-02-05T10:07:18.720412Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we can import our training dataset.\n\nš The `train` file below is an exact copy of the one within the `tensorflow-great-barrier-reef` dataset and a few more columns that were created [in my first notebook](https://www.kaggle.com/andradaolteanu/greatbarrierreef-full-guide-to-bboxaugmentation):\n* **`no_annotations`**: number of bboxes found within the image\n* **`path`**: full path to the `tensorflow-great-barrier-reef` image\n* **`f_annotations`**: formated annotations created for image augmentation\n* **`path_images` & `path_labels`**: full paths to the directories we have created above\n* **`width` & `height`**: the same all over, represent the metrics for the image\n* **`coco_bbox`**: the simplified version of the `annotations` column. The *COCO format* is one of the many ways to annotate a bounding box and it has the format `[x, y, width, height]`.","metadata":{}},{"cell_type":"code","source":"# Import the prepped train dataset\ntrain = pd.read_csv(\"../input/2021-greatbarrierreef-prep-data/train.csv\")\n# Remove all images that have no bounding box (removing ~80% of data)\ntrain = train[train[\"no_annotations\"]>0].reset_index(drop=True)\n\ntrain.sample(3, random_state=24)","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:07:18.730966Z","iopub.execute_input":"2022-02-05T10:07:18.733155Z","iopub.status.idle":"2022-02-05T10:07:19.026285Z","shell.execute_reply.started":"2022-02-05T10:07:18.733080Z","shell.execute_reply":"2022-02-05T10:07:19.025240Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Step 2. Copy Images & Labels\n\nOnce we have our dataset and folders, we can proceed with the next step. Now we will copy the needed data (the images to train on and the labels) into these directories. We do this so we can have *writing access* on it.\n\n## I. The Images\n\n**Copy** from `../input/tensorflow-great-barrier-reef/train_images` to `../images`.\n\n> **Note**: `shutil` library is used to copy files from one place to another.","metadata":{}},{"cell_type":"code","source":"# Populate the ../images folder\n\nfor path in tqdm(train[\"path\"].tolist()):\n split_path = path.split(\"/\")\n\n # Retrieve the video id (0, 1, 2) and its frame number\n video_id = split_path[-2]\n video_frame = split_path[-1]\n\n # Create new image path\n path_image = f\"../images/{video_id}_{video_frame}\"\n \n # Copy file from source (competition data) to destination (our new folder)\n shutil.copy(src=path, dst=path_image)","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:07:19.032523Z","iopub.execute_input":"2022-02-05T10:07:19.035185Z","iopub.status.idle":"2022-02-05T10:08:12.755319Z","shell.execute_reply.started":"2022-02-05T10:07:19.035123Z","shell.execute_reply":"2022-02-05T10:08:12.754266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Glimpse of images folder now:\nprint(color.S+\"Sample of 3 images from the ../images/ folder:\"+color.E, os.listdir(\"../images\")[:3])\n\nplt.figure(figsize=(10, 10))\nimg_sample = cv2.imread(\"../images/video_1_6258.jpg\")\nimg_sample = cv2.cvtColor(img_sample, cv2.COLOR_BGR2RGB)\nplt.imshow(img_sample)\nplt.axis(\"off\");","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:08:12.756781Z","iopub.execute_input":"2022-02-05T10:08:12.757412Z","iopub.status.idle":"2022-02-05T10:08:13.360131Z","shell.execute_reply.started":"2022-02-05T10:08:12.757356Z","shell.execute_reply":"2022-02-05T10:08:13.358757Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## II. The Labels\n\n### COCO2YOLO\n\nBefore copying the labels we need to create a function that **converts** the COCO bboxes to YOLO bboxes.\n\nHence, we need to go from `[xmin, ymin, w, h]` to the corresponding yolo format `[xmid, ymid, w, h]`.\n\n![](\"https://i0.wp.com/prabhjotkaurgosal.com/wp-content/uploads/2021/03/image.png?resize=1536%2C419&ssl=1\")
\n\n*Source: [here](https://prabhjotkaurgosal.com/weekly-learnings/weekly-learning-blogs/)*","metadata":{}},{"cell_type":"code","source":"def coco2yolo(image_height, image_width, bboxes):\n \"\"\"\n Converts a coco annotation format [xmin, ymin, w, h] to \n the corresponding yolo format [xmid, ymid, w, h]\n \n image_height: height of the original image\n image_width: width of the original image\n bboxes: coco boxes to be converted\n return :: \n \n inspo: https://www.kaggle.com/awsaf49/great-barrier-reef-yolov5-train\n \"\"\"\n \n bboxes = np.array(bboxes).astype(float)\n \n # Normalize xmin, w\n bboxes[:, [0, 2]]= bboxes[:, [0, 2]]/ image_width\n # Normalize ymin, h\n bboxes[:, [1, 3]]= bboxes[:, [1, 3]]/ image_height\n \n # Converstion (xmin, ymin) => (xmid, ymid)\n bboxes[:, [0, 1]] = bboxes[:, [0, 1]] + bboxes[:, [2, 3]]/2\n \n # Clip values (between 0 and 1)\n bboxes = np.clip(bboxes, a_min=0, a_max=1)\n \n return bboxes","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:08:13.361507Z","iopub.execute_input":"2022-02-05T10:08:13.361801Z","iopub.status.idle":"2022-02-05T10:08:14.479973Z","shell.execute_reply.started":"2022-02-05T10:08:13.361761Z","shell.execute_reply":"2022-02-05T10:08:14.478835Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# --- Example ---\nbbox_example = [[559, 213, 50, 32], [679, 223, 10, 100]]\n\nprint(color.S+\"From COCO: \"+color.E, bbox_example)\nprint(color.S+\"to YOLO:\"+color.E, \n coco2yolo(image_height=720, \n image_width=1280, \n bboxes=bbox_example))","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:08:14.483502Z","iopub.execute_input":"2022-02-05T10:08:14.484203Z","iopub.status.idle":"2022-02-05T10:08:14.497214Z","shell.execute_reply.started":"2022-02-05T10:08:14.484155Z","shell.execute_reply":"2022-02-05T10:08:14.495911Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Copy Labels\n\nThe **YOLO** template requires a little bit more than just the new YOLO bboxes. You can [read this article](https://towardsdatascience.com/image-data-labelling-and-annotation-everything-you-need-to-know-86ede6c684b1) to find out a little bit more about how the process works.\n\nš **The Summary**: In the YOLO labeling format, a `.txt` file with the same name is created for each image file in the same directory. Each `.txt` file contains the annotations for the corresponding image file, that is:\n* *object class* - not applicable in our case, so it will be always set to 0\n* *YOLO bbox* - the YOLO bboxes we have just created","metadata":{}},{"cell_type":"code","source":"# Populate the ../labels folder\nyolo_bboxes = []\n\nfor k in tqdm(range(len(train))):\n \n row_data = train.iloc[k, :]\n height = row_data[\"height\"]\n width = row_data[\"width\"]\n coco_bbox = eval(row_data[\"coco_bbox\"])\n len_bbox = row_data[\"no_annotations\"]\n \n # Create file and write in it\n with open(row_data[\"path_labels\"], 'w') as file:\n \n # In case there is an image with no present annotation\n if len_bbox == 0: \n file.write(\"\")\n continue\n \n # Convert coco format to yolo format\n yolo_bbox = coco2yolo(height, width, coco_bbox)\n yolo_bboxes.append(yolo_bbox)\n \n # Write annotations in file\n for i in range(len_bbox):\n annot = [\"0\"] + \\\n yolo_bbox[i].astype(str).tolist() + \\\n ([\"\"] if i+1 == len_bbox else [\"\\n\"])\n \n annot = \" \".join(annot).strip()\n file.write(annot)\n \n\n# Add yolo boxes to dataframe\ntrain[\"yolo_bbox\"] = yolo_bboxes","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:08:14.499157Z","iopub.execute_input":"2022-02-05T10:08:14.499555Z","iopub.status.idle":"2022-02-05T10:08:17.713219Z","shell.execute_reply.started":"2022-02-05T10:08:14.499497Z","shell.execute_reply":"2022-02-05T10:08:17.712112Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Glimpse of labels folder now:\nprint(color.S+\"Sample of 3 labels from the ../labels/ folder:\"+color.E, os.listdir(\"../labels\")[:3], \"\\n\")\n\n# Let's read the files\nf1 = open('../labels/video_1_4238.txt', 'r')\nf2 = open('../labels/video_1_5315.txt', 'r')\nf3 = open('../labels/video_0_1006.txt', 'r')\n\n# How the .txt files look?\nprint(color.S+\"File1: \"+color.E, f1.read())\nprint(color.S+\"File2: \"+color.E, f2.read())\nprint(color.S+\"File3: \"+color.E, f3.read())","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:08:17.715072Z","iopub.execute_input":"2022-02-05T10:08:17.715564Z","iopub.status.idle":"2022-02-05T10:08:17.736229Z","shell.execute_reply.started":"2022-02-05T10:08:17.715472Z","shell.execute_reply":"2022-02-05T10:08:17.735018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## III. Did we do a good job?\n\nWe have created 2 folders: the images and the corresponding labels (annotations).\n\nLet's test a few images to see if the look good.","metadata":{}},{"cell_type":"code","source":"# Retrieve a sample of data\nimages = os.listdir(\"/kaggle/images\")[6:12]\nvid_id = [im.split(\"_\")[1] for im in images]\nseq_id = [im.split(\"_\")[2].split(\".\")[0] for im in images]\n\n# Plot\nfig, axs = plt.subplots(2, 3, figsize=(23, 10))\naxs = axs.flatten()\nfig.suptitle(f\"Sample of images and YOLO bounding boxes\", fontsize = 20)\n\nfor k in range(6):\n \n # Get the data\n im = cv2.imread(f\"/kaggle/images/{images[k]}\")\n im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)\n dh, dw, _ = im.shape\n txt = open(f\"/kaggle/labels/video_{vid_id[k]}_{seq_id[k]}.txt\", \"r\").read().split(\" \")[1:]\n no_boxes = int(len(txt)/4)\n \n # Draw boxes\n i = 0\n while i < no_boxes:\n i = i+4\n box = txt[:i][-4:]\n \n # Src: https://github.com/pjreddie/darknet/blob/810d7f797bdb2f021dbe65d2524c2ff6b8ab5c8b/src/image.c#L283-L291\n # from YOLO to COCO\n x, y, w, h = box\n x, y, w, h = float(x), float(y), float(w), float(h)\n\n l = int((x - w / 2) * dw)\n r = int((x + w / 2) * dw)\n t = int((y - h / 2) * dh)\n b = int((y + h / 2) * dh)\n\n if l < 0: l = 0\n if r > dw - 1: r = dw - 1\n if t < 0: t = 0\n if b > dh - 1: b = dh - 1\n\n cv2.rectangle(im, (l, t), (r, b), (255,0,0), 3)\n \n # Show image with bboxes\n axs[k].set_title(f\"Sample {k}\", fontsize = 14)\n axs[k].imshow(im)\n axs[k].set_axis_off()\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:08:17.738477Z","iopub.execute_input":"2022-02-05T10:08:17.739003Z","iopub.status.idle":"2022-02-05T10:08:19.736552Z","shell.execute_reply.started":"2022-02-05T10:08:17.738956Z","shell.execute_reply":"2022-02-05T10:08:19.735444Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Step 3. YOLO Configuration\n\n## I. Splitting the Data\n\nAs I have seen within discussions that it is best to group the data by taking into account that the images are actually *videos*. That being said, you don't want to train you model on image `i` and then test it on image `i+1`, as this could be translated to **data leakage**.\n\nIt's like showing a child a cat now and then moving it a little bit to the left and asking the child what animal it is. Of course they'll respond correctly! But if you show a child a cat and after an hour you show another cat in another environment ... then is when you'll actually test their capabilities of learning.\n\nš Hence, I have splitted the data simply into 2 parts:\n* `train_data`: images from video_0 (2,143 observations) and video_2 (677 observations)\n* `test_data`: images from video_1 (2,099 observations)","metadata":{}},{"cell_type":"code","source":"# Use Video 0 and 2 for training and 1 for validation\ntrain_data = train[train[\"video_id\"].isin([0, 2])]\ntest_data = train[train[\"video_id\"].isin([1])]\n\n# Get path to images & labels\ntrain_images = list(train_data[\"path_images\"])\ntrain_labels = list(train_data[\"path_labels\"])\n\ntest_images = list(test_data[\"path_images\"])\ntest_labels = list(test_data[\"path_labels\"])\n\nprint(color.S+\"Train Length:\"+color.E, len(train_data), \"\\n\" +\n color.S+\"Test Length:\"+color.E, len(test_data))","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:08:19.739566Z","iopub.execute_input":"2022-02-05T10:08:19.739844Z","iopub.status.idle":"2022-02-05T10:08:19.761789Z","shell.execute_reply.started":"2022-02-05T10:08:19.739806Z","shell.execute_reply":"2022-02-05T10:08:19.760363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## II. Configuration Setup\n\nThis part will require us to:\n* Create a \"../working/train_images.txt\"\n* Create a \"../working/test_images.txt\" - these will be populated later\n* Create a `yaml` configuration file","metadata":{}},{"cell_type":"code","source":"print(color.S+\"../working BEFORE:\"+color.E, os.listdir(\"../working\"))\n\n# Create train and test path data\nwith open(\"../working/train_images.txt\", \"w\") as file:\n for path in train_images:\n file.write(path + \"\\n\")\n \nwith open(\"../working/test_images.txt\", \"w\") as file:\n for path in test_images:\n file.write(path + \"\\n\")\n\n\n# Create configuration\nconfig = {'path': '/kaggle/working',\n 'train': '/kaggle/working/train_images.txt',\n 'val': '/kaggle/working/test_images.txt',\n 'nc': 1,\n 'names': ['cots']}\n\nwith open(\"../working/cots.yaml\", \"w\") as file:\n yaml.dump(config, file, default_flow_style=False)\n\n \nprint(color.S+\"../working AFTER:\"+color.E, os.listdir(\"../working\"))","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:08:19.763580Z","iopub.execute_input":"2022-02-05T10:08:19.764167Z","iopub.status.idle":"2022-02-05T10:08:19.783737Z","shell.execute_reply.started":"2022-02-05T10:08:19.764126Z","shell.execute_reply":"2022-02-05T10:08:19.782645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Step 4. YOLOv5 Training\n\n## I. YOLOv5\n\nYOLO's first model was released in 2016, followed by YOLOv2 in 2017 and YOLOv3 in 2018. In 2020 [Joseph Redmon](https://machinelearningknowledge.ai/introduction-to-yolov5-object-detection-with-tutorial/#Introduction) stepped out from the project and his work was further improved by Alexey Bochkovskiy who produced YOLOv4 in 2020.\n\n**YOLOv5** is the next controversial member of the YOLO family released in 2020 by the company Ultranytics just a few days after YOLOv4 ([source here](https://machinelearningknowledge.ai/introduction-to-yolov5-object-detection-with-tutorial/#Introduction)). It is controversial because there has never been any paper released to back up the model, nevertheless it works!\n\n![](\"https://machinelearningknowledge.ai/wp-content/uploads/2021/06/YOLOv5-Architecture.jpg\")
\n\n*[Source here](https://www.researchgate.net/publication/349299852_A_Forest_Fire_Detection_System_Based_on_Ensemble_Learning)*\n\n## II. YOLOv5 Setup\n\nTo use the model we need to have the following:\n* Yolov5 Repository ([available in this dataset by Awsaf](https://www.kaggle.com/awsaf49/yolov5-lib-ds))\n* Python 3\n* PyTorch\n* CUDA\n\n> **Note**: put `/kaggle/` and not `../` otherwise won't work.","metadata":{}},{"cell_type":"code","source":"# ---> YOLOv5 install <---\n%cd /kaggle/working \n!cp -r /kaggle/input/yolov5-lib-ds /kaggle/working/yolov5 \n%cd yolov5 \n%pip install -qr requirements.txt \n\nfrom yolov5 import utils\ndisplay = utils.notebook_init()","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:08:19.788007Z","iopub.execute_input":"2022-02-05T10:08:19.788589Z","iopub.status.idle":"2022-02-05T10:08:35.442185Z","shell.execute_reply.started":"2022-02-05T10:08:19.788549Z","shell.execute_reply":"2022-02-05T10:08:35.440947Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## III. Training\n\n* š **Note**: YOLOv5 **connects automatically to your W&B account** and tracks the runs and progress there, so you do not need to log in anything during training.\n\nš Within the training cell:\n* specify the dataset - `cots.yaml`\n* batch size\n* image size\n* pretrained yolov5 weights (`yolov5s.pt`, `yolov5m.pt` etc.)\n\n**There are multiple models you could try:**\n\n![](\"https://github.com/ultralytics/yolov5/releases/download/v1.0/model_comparison.png\")
\n\n*[Source here](https://docs.ultralytics.com/tutorials/train-custom-datasets/) - [full table with all available options here](https://github.com/ultralytics/yolov5#pretrained-checkpoints)*","metadata":{}},{"cell_type":"code","source":"# --- PARAMETERS ---\n# These are just small samples, so the notebook runs faster\nSIZE = 500\nBATCH_SIZE = 4\nEPOCHS = 3\nMODEL = \"yolov5s\"\nWORKERS = 1\nPROJECT = \"GreatBarrierReef\"\nRUN_NAME = f\"{MODEL}_size{SIZE}_epochs{EPOCHS}_batch{BATCH_SIZE}_simple\"\n# ------------------","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:08:35.444169Z","iopub.execute_input":"2022-02-05T10:08:35.445390Z","iopub.status.idle":"2022-02-05T10:08:35.453515Z","shell.execute_reply.started":"2022-02-05T10:08:35.445311Z","shell.execute_reply":"2022-02-05T10:08:35.452235Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Training - train.py can be found in yolov5 directory\n!python train.py --img {SIZE}\\\n --batch {BATCH_SIZE}\\\n --epochs {EPOCHS}\\\n --data /kaggle/working/cots.yaml\\\n --weights {MODEL}.pt\\\n --workers {WORKERS}\\\n --project {PROJECT}\\\n --name {RUN_NAME}\\\n --exist-ok","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-02-05T10:08:35.455158Z","iopub.execute_input":"2022-02-05T10:08:35.455709Z","iopub.status.idle":"2022-02-05T10:31:01.434578Z","shell.execute_reply.started":"2022-02-05T10:08:35.455654Z","shell.execute_reply":"2022-02-05T10:31:01.433459Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## IV. Inspecting the results\n\nAll training results are saved to `../working/yolov5/runs/train/` with incrementing run directories (first run is `exp`, then `exp2`, `exp3` and so on).\n\n> š **Note**: in this notebooks, `runs` is actually the folder `GreatBarrierReef`, as I am saving the logs during training within [my personal Dashboard for this competition here](https://wandb.ai/andrada/GreatBarrierReef?workspace=user-andrada). This way I can properly *name* the experiments - so, instead of having exp1, exp2 etc, I can have a proper name that will better indicate the experiment I am making. Also, **all data in this folder can be viewed in the W&B Dashboard**.\n\n![](\"https://i.imgur.com/n8elExY.png\")
\n\nš Below it's a view of the new files created after training:","metadata":{}},{"cell_type":"code","source":"# Run details\nos.listdir(f\"{PROJECT}/{RUN_NAME}\")","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:31:01.437710Z","iopub.execute_input":"2022-02-05T10:31:01.438529Z","iopub.status.idle":"2022-02-05T10:31:01.450426Z","shell.execute_reply.started":"2022-02-05T10:31:01.438479Z","shell.execute_reply":"2022-02-05T10:31:01.449060Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here is a sneak peak of how the folder looks locally on my workstation:\n\n![](\"https://i.imgur.com/V5TzfyD.png\")
\n\nš I went ahead and saved the trained model here: `../input/2021-greatbarrierreef-prep-data`\n\nYou can also find it in [my dataset on this competition](https://www.kaggle.com/andradaolteanu/2021-greatbarrierreef-prep-data) in the folder [output].","metadata":{}},{"cell_type":"code","source":"# Remove training data files\n!rm -r '/kaggle/images'\n!rm -r '/kaggle/labels'","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-02-05T10:31:01.452475Z","iopub.execute_input":"2022-02-05T10:31:01.452896Z","iopub.status.idle":"2022-02-05T10:31:04.399594Z","shell.execute_reply.started":"2022-02-05T10:31:01.452832Z","shell.execute_reply":"2022-02-05T10:31:04.398236Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Step 5. Inference\n\n## I. Competition metric\n\nš **F2** metric is computed based on:\n* **precision**: which looks at how accurately we have identified the positives (the false positives should be minimum)\n* **recall**: which looks at how accurately we have identified the true positives and true negaives in general (surprising as many as possible)\n\n![](\"https://i.imgur.com/tfjRMDA.png\")
\n\nThis competiton uses [the F2 Score at different intersection over union (IoU) thresholds](https://www.kaggle.com/c/tensorflow-great-barrier-reef/overview/evaluation). This adjusted F2 puts in balance and is **in favor for recall rather than precision**, meaning that we want to have as many true cases of COTS catched, so we don't mind a few false positives here and there as well.\n\n> š **Note**: Each predicted bounding box should have a *confidence level*, meaning that we need to provide a *certainty* that within the bounding box is actually a COTS or not.\n\n## II. Loading the model\n\n> š **Note**: I will use [this dataset](reef_baseline_fold12) by [sheep](https://www.kaggle.com/steamedsheep) to do the inference. š","metadata":{}},{"cell_type":"code","source":"# Change our position within the directory back\n%cd /kaggle/working","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:31:04.402637Z","iopub.execute_input":"2022-02-05T10:31:04.402975Z","iopub.status.idle":"2022-02-05T10:31:04.410656Z","shell.execute_reply.started":"2022-02-05T10:31:04.402929Z","shell.execute_reply":"2022-02-05T10:31:04.409806Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# --- Trained Model ---\nMODEL_PATH = \"../input/reef-baseline-fold12/l6_3600_uflip_vm5_f12_up/f1/best.pt\"\n\n# Load the model\nmodel = torch.hub.load(\"../input/yolov5-lib-ds\", \"custom\",\n path=MODEL_PATH,\n source='local', force_reload=True)\n\n# BoundingBox Confidence\nmodel.conf = 0.01\n# Intersection Over Union\nmodel.iou = 0.5","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:31:04.412058Z","iopub.execute_input":"2022-02-05T10:31:04.412393Z","iopub.status.idle":"2022-02-05T10:31:09.210919Z","shell.execute_reply.started":"2022-02-05T10:31:04.412342Z","shell.execute_reply":"2022-02-05T10:31:09.209915Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create Ultralytics directory\n!mkdir -p /root/.config/Ultralytics\n# Copy folder to root\n!cp /kaggle/input/yolov5-font/Arial.ttf /root/.config/Ultralytics/","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:31:09.213191Z","iopub.execute_input":"2022-02-05T10:31:09.213537Z","iopub.status.idle":"2022-02-05T10:31:10.854173Z","shell.execute_reply.started":"2022-02-05T10:31:09.213493Z","shell.execute_reply":"2022-02-05T10:31:10.852922Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## III. Prediction","metadata":{}},{"cell_type":"code","source":"import greatbarrierreef\n\n# Initialize the environment\nenv = greatbarrierreef.make_env()\n# Iterator that loops through the submission dataset\n# !!! you can run this cell only once\niter_test = env.iter_test()","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:31:10.856696Z","iopub.execute_input":"2022-02-05T10:31:10.857011Z","iopub.status.idle":"2022-02-05T10:31:10.884707Z","shell.execute_reply.started":"2022-02-05T10:31:10.856978Z","shell.execute_reply":"2022-02-05T10:31:10.883725Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"š Looking at the requirements, the sample prediction must look as follows: `sample_prediction_df['annotations'] = '0.5 0 0 100 100'`. Hence, the submitted bounding box should have the format `'conf x y width height'`.\n\nIf there are multiple boxes in one image they would look like: `'conf1 x1 y1 width1 height1 conf2 x2 y2 width2 height2'`.","metadata":{}},{"cell_type":"code","source":"# !!! you can run this cell only once\n\n# Loop through the test file\nfor k, (image, sample_prediction_df) in enumerate(tqdm(iter_test)):\n \n annotation = \"\"\n prediction = model(image, size=3600, augment=True)\n print(color.S+\"Prediction Object:\"+color.E, prediction.pandas())\n print(color.S+\"Bounding Boxes:\"+color.E, prediction.pandas().xyxy[0])\n print(color.S+\"Shape:\"+color.E, prediction.pandas().xyxy[0].shape[0])\n \n if prediction.pandas().xyxy[0].shape[0] == 0:\n annotation = \"\"\n else:\n for k, row in prediction.pandas().xyxy[0].iterrows():\n if row.confidence > 0.15:\n conf = row.confidence\n x = int(row.xmin)\n y = int(row.ymin)\n width = int(row.xmax-row.xmin)\n height = int(row.ymax-row.ymin)\n annotation += \"{} {} {} {} {}\".format(conf, x, y, width, height)\n \n sample_prediction_df['annotations'] = annotation.strip(' ')\n \n # Register your predictions\n env.predict(sample_prediction_df)","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:31:10.886815Z","iopub.execute_input":"2022-02-05T10:31:10.887105Z","iopub.status.idle":"2022-02-05T10:31:11.812242Z","shell.execute_reply.started":"2022-02-05T10:31:10.887052Z","shell.execute_reply":"2022-02-05T10:31:11.811142Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](\"https://i.imgur.com/0cx4xXI.png\")
\n\n### š W&B Dashboard\n\n> My [W&B Dashboard](https://wandb.ai/andrada/GreatBarrierReef/workspace?workspace=user-andrada).\n\n\n\n![](\"https://i.imgur.com/knxTRkO.png\")
\n\n### My Specs\n\n* š„ Z8 G4 Workstation\n* š¾ 2 CPUs & 96GB Memory\n* š® NVIDIA Quadro RTX 8000\n* š» Zbook Studio G7 on the go","metadata":{}}]}