{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"This is a starter kernel to train a YOLOv5 model on Tensorflow - Help Protect the Great Barrier Reef dataset. Given an input image the task is to find the presence of crown-of-thorns starfish using bounding box detection.\n\n### Other works in this competition\n\nYou can check out my visualization kernel [here](https://www.kaggle.com/ayuraj/visualize-the-cots-interactively-with-w-b?scriptVersionId=80596542).\n\n# 🖼️ What is YOLOv5?\n\nYOLO an acronym for 'You only look once', is an object detection algorithm that divides images into a grid system. Each cell in the grid is responsible for detecting objects within itself.\n\n[Ultralytics' YOLOv5](https://github.com/ultralytics/yolov5) (\"You Only Look Once\") model family enables real-time object detection with convolutional neural networks.\n\n# 🦄 What is Weights and Biases?\n\n[Weights & Biases](https://wandb.ai/site) (W&B) is a set of machine learning tools that helps you build better models faster. Check out Experiment Tracking with Weights and Biases to learn more.\nWeights & Biases is directly integrated into YOLOv5, providing experiment metric tracking, model and dataset versioning, rich model prediction visualization, and more. You can learn more about W&B in this [kernel](https://www.kaggle.com/ayuraj/experiment-tracking-with-weights-and-biases).","metadata":{}},{"cell_type":"markdown","source":"# ☀️ Imports and Setup\n\nAccording to the official Train Custom Data guide, YOLOv5 requires a certain directory structure.\n\n```\n/parent_folder\n    /dataset\n         /images\n         /labels\n    /yolov5\n```\n\n* We thus will create a /tmp directory.\n* Download YOLOv5 repository and pip install the required dependencies.\n* Install the latest version of W&B and login with your wandb account. You can create your free W&B account [here](https://wandb.ai/signup).","metadata":{}},{"cell_type":"code","source":"%cd ../\n!mkdir tmp\n%cd tmp","metadata":{"execution":{"iopub.status.busy":"2021-11-23T22:42:34.328710Z","iopub.execute_input":"2021-11-23T22:42:34.329367Z","iopub.status.idle":"2021-11-23T22:42:35.012478Z","shell.execute_reply.started":"2021-11-23T22:42:34.329268Z","shell.execute_reply":"2021-11-23T22:42:35.011659Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Download YOLOv5\n!git clone https://github.com/ultralytics/yolov5  # clone repo\n%cd yolov5\n# Install dependencies\n%pip install -qr requirements.txt  # install dependencies\n\n%cd ../\nimport torch\nprint(f\"Setup complete. Using torch {torch.__version__} ({torch.cuda.get_device_properties(0).name if torch.cuda.is_available() else 'CPU'})\")","metadata":{"execution":{"iopub.status.busy":"2021-11-23T22:42:36.283945Z","iopub.execute_input":"2021-11-23T22:42:36.284521Z","iopub.status.idle":"2021-11-23T22:42:50.043064Z","shell.execute_reply.started":"2021-11-23T22:42:36.284481Z","shell.execute_reply":"2021-11-23T22:42:50.042205Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## How to login to W&B?\n\nThere are two ways you can login to W&B in a Kaggle kernel setting:\n\n* Run a cell with `wandb.login()`. It will ask for the API key, which you can copy + paste in. This is ideal if you Quick Save your kernel.\n\n* You can also use Kaggle Secrets to store your API key and use the code snippet below to login. If you are not familiar with Kaggle Secrets check this [forum post](https://www.kaggle.com/product-feedback/114053). This is ideal if you do Run and Save All.\n\n```\nfrom kaggle_secrets import UserSecretsClient\n\nuser_secrets = UserSecretsClient()\n\n# I have saved my API token with \"wandb_api\" as Label. \n# If you use some other Label make sure to change the same below. \nwandb_api = user_secrets.get_secret(\"wandb_api\") \n\nwandb.login(key=wandb_api)\n\n```\n\nMore on W&B login [here](https://docs.wandb.ai/ref/cli/wandb-login).\n","metadata":{}},{"cell_type":"code","source":"# Install W&B \n# !pip install -q --upgrade wandb\n\n# Login \nimport wandb\nprint(wandb.__version__)\nwandb.login(anonymous='must')","metadata":{"execution":{"iopub.status.busy":"2021-11-23T22:42:50.044938Z","iopub.execute_input":"2021-11-23T22:42:50.045942Z","iopub.status.idle":"2021-11-23T22:42:52.025451Z","shell.execute_reply.started":"2021-11-23T22:42:50.045896Z","shell.execute_reply":"2021-11-23T22:42:52.024605Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n📍 Note: W&B comes pre-installed with Kaggle kernel but to ensure the lastest version of W&B use pip install.\n\n","metadata":{}},{"cell_type":"code","source":"# Necessary/extra dependencies. \nimport os\nimport gc\nimport ast\nimport cv2\nimport numpy as np\nimport pandas as pd\nfrom tqdm import tqdm\ntqdm.pandas()\nfrom shutil import copyfile\nimport matplotlib.pyplot as plt\nfrom sklearn.model_selection import train_test_split","metadata":{"execution":{"iopub.status.busy":"2021-11-23T22:42:52.027417Z","iopub.execute_input":"2021-11-23T22:42:52.027969Z","iopub.status.idle":"2021-11-23T22:42:52.882974Z","shell.execute_reply.started":"2021-11-23T22:42:52.027925Z","shell.execute_reply":"2021-11-23T22:42:52.882094Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🦆 Hyperparameters¶\n","metadata":{}},{"cell_type":"code","source":"%cd ../\nTRAIN_PATH = 'input/tensorflow-great-barrier-reef/train_images'\nIMG_SIZE = 256\nBATCH_SIZE = 16\nEPOCHS = 10","metadata":{"execution":{"iopub.status.busy":"2021-11-23T22:42:52.885243Z","iopub.execute_input":"2021-11-23T22:42:52.885505Z","iopub.status.idle":"2021-11-23T22:42:52.893794Z","shell.execute_reply.started":"2021-11-23T22:42:52.885471Z","shell.execute_reply":"2021-11-23T22:42:52.892941Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🔨 Prepare Dataset\n\nThis is the most important section when it comes to training an object detector with YOLOv5. The directory structure, bounding box format, etc must be in the correct order. This section builds every piece needed to train a YOLOv5 model.\n\n* Find the number of images with annotations and remove the ones without annotations.\n* Create train-validation split.\n* Create required /dataset folder structure and more the images to that folder.\n* Create `data.yaml` file needed to train the model.\n* Create bounding box coordinates in the required YOLO format.","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv('input/tensorflow-great-barrier-reef/train.csv')\nprint('Number of training images: ', len(df))\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-23T22:42:52.895295Z","iopub.execute_input":"2021-11-23T22:42:52.895667Z","iopub.status.idle":"2021-11-23T22:42:52.961294Z","shell.execute_reply.started":"2021-11-23T22:42:52.895629Z","shell.execute_reply":"2021-11-23T22:42:52.960594Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"📍 Notes:\n\n* There are three unique video_ids. These are 0, 1, and 2 and are three distinct directories (video_0, video_1, and video_2).\n* There are 20 unique sequences. ","metadata":{}},{"cell_type":"code","source":"def add_path(row):\n    return f\"{TRAIN_PATH}/video_{row.video_id}/{row.video_frame}.jpg\"\n\ndef num_boxes(annotations):\n    annotations = ast.literal_eval(annotations)\n    return len(annotations)\n\n# Add path\ndf['path'] = df.apply(lambda row: add_path(row), axis=1)\n# Find number of annotations per image\ndf['num_bbox'] = df['annotations'].apply(lambda x: num_boxes(x))\n\ndata = (df.num_bbox>0).value_counts()/len(df)*100\nprint(f\"Number of images without bboxes: {data[0]:0.2f}% | with bboxes: {data[1]:0.2f}%\")","metadata":{"execution":{"iopub.status.busy":"2021-11-23T22:42:52.962740Z","iopub.execute_input":"2021-11-23T22:42:52.963230Z","iopub.status.idle":"2021-11-23T22:42:53.855364Z","shell.execute_reply.started":"2021-11-23T22:42:52.963195Z","shell.execute_reply":"2021-11-23T22:42:53.854473Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Remove images without annotations. \ndf = df.query(\"num_bbox>0\")\nprint(f\"Number of images with annotations: {len(df)}\")\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-23T22:42:53.856636Z","iopub.execute_input":"2021-11-23T22:42:53.856941Z","iopub.status.idle":"2021-11-23T22:42:53.887528Z","shell.execute_reply.started":"2021-11-23T22:42:53.856903Z","shell.execute_reply":"2021-11-23T22:42:53.886642Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create train and validation split\ntrain_df, valid_df, = train_test_split(df, test_size=0.15, random_state=42)\n\ntrain_df.loc[:, 'split'] = 'train'\nvalid_df.loc[:, 'split'] = 'valid'\n\ndf = pd.concat([train_df, valid_df]).reset_index(drop=True)\nprint(f\"Number of train images: {len(train_df)}, validation images: {len(valid_df)}\")","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-11-23T22:42:53.888700Z","iopub.execute_input":"2021-11-23T22:42:53.889400Z","iopub.status.idle":"2021-11-23T22:42:53.908247Z","shell.execute_reply.started":"2021-11-23T22:42:53.889359Z","shell.execute_reply":"2021-11-23T22:42:53.907439Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🍚 Prepare Required Folder Structure\n\nThe required folder structure for the dataset directory is:\n\n```\n/parent_folder\n    /dataset\n         /images\n             /train\n             /val\n         /labels\n             /train\n             /val\n    /yolov5\n```\n\nNote that I have named the directory `barrier_reef`.\n","metadata":{}},{"cell_type":"code","source":"def add_new_path(row):\n    return f\"tmp/barrier_reef/dataset/images/{row.split}/{row.video_frame}.jpg\"\n    \n# Add new path\ndf['new_path'] = df.apply(lambda row: add_new_path(row), axis=1)","metadata":{"execution":{"iopub.status.busy":"2021-11-23T22:42:53.909747Z","iopub.execute_input":"2021-11-23T22:42:53.910176Z","iopub.status.idle":"2021-11-23T22:42:54.041496Z","shell.execute_reply.started":"2021-11-23T22:42:53.910139Z","shell.execute_reply":"2021-11-23T22:42:54.040685Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def copy_file(row):\n    os.makedirs(os.path.dirname(row.new_path), exist_ok=True)\n    copyfile(row.path, row.new_path)\n\n_ = df.progress_apply(lambda row: copy_file(row), axis=1)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-11-23T22:42:54.043886Z","iopub.execute_input":"2021-11-23T22:42:54.044147Z","iopub.status.idle":"2021-11-23T22:43:46.473885Z","shell.execute_reply.started":"2021-11-23T22:42:54.044107Z","shell.execute_reply":"2021-11-23T22:43:46.473121Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🍜 Create .YAML file\n\nThe `data.yaml`, is the dataset configuration file that defines\n\n- an \"optional\" download command/URL for auto-downloading,\n- a path to a directory of training images (or path to a *.txt file with a list of training images),\n- a path to a directory of validation images (or path to a *.txt file with a list of validation images), \n- the number of classes, \n- a list of class names\n\n📍 Important: There is just one class that's why nc=1. <br>\n📍 Note: The `data.yaml` is created in the yolov5/data directory as required.","metadata":{}},{"cell_type":"code","source":"# Create .yaml file \nimport yaml\n\ndata_yaml = dict(\n    train = '../barrier_reef/images/train',\n    val = '../barrier_reef/images/valid',\n    nc = 1,\n    names = ['cots']\n)\n\n# Note that I am creating the file in the yolov5/data/ directory.\nwith open('tmp/yolov5/data/data.yaml', 'w') as outfile:\n    yaml.dump(data_yaml, outfile, default_flow_style=True)\n    \n%cat tmp/yolov5/data/data.yaml","metadata":{"execution":{"iopub.status.busy":"2021-11-23T22:43:46.475098Z","iopub.execute_input":"2021-11-23T22:43:46.475790Z","iopub.status.idle":"2021-11-23T22:43:47.156286Z","shell.execute_reply.started":"2021-11-23T22:43:46.475736Z","shell.execute_reply":"2021-11-23T22:43:47.155408Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🍮 Prepare Bounding Box Coordinated for YOLOv5\n\nFor every image with bounding box(es) a `.txt` file with the same name as the image will be created in the format shown below:\n\n* One row per object.\n* Each row is in `class x_center y_center width height` format.\n* Box coordinates must be in normalized xywh format (from 0 - 1). We can normalize by the boxes in pixels by dividing `x_center` and `width` by image width, and `y_center` and `height` by image `height`.\n* Class numbers are zero-indexed (start from 0).\n\n📍 Note: We don't have to remove the images without bounding boxes from the training or validation sets. But in our case we are not considering images without bboxes.","metadata":{}},{"cell_type":"code","source":"IMG_WIDTH, IMG_HEIGHT = 1280, 720\n\ndef get_yolo_format_bbox(img_w, img_h, box):\n    \"\"\"\n    Convert the bounding boxes in YOLO format.\n    \n    Input:\n    img_w - Original/Scaled image width\n    img_h - Original/Scaled image height\n    box - Bounding box coordinates in the format, \"x_min, y_min, width, height\"\n    \n    Output:\n    Return YOLO formatted bounding box coordinates, \"x_center y_center width height\".\n    \"\"\"\n    w = box['width'] # width \n    h = box['height'] # height\n    xc = box['x'] + int(np.round(w/2)) # xmin + width/2\n    yc = box['y'] + int(np.round(h/2)) # ymin + height/2\n\n    return [xc/img_w, yc/img_h, w/img_w, h/img_h] # x_center y_center width height\n    \n\n# Iterate over each row and write the labels and bbox coordinates to a .txt file. \nfor i in tqdm(range(len(df))):\n    row = df.loc[i]\n    annotations = ast.literal_eval(row.annotations)\n    bboxes = []\n    for bbox in annotations:\n        # get bbox in YOLO format\n        bbox = get_yolo_format_bbox(IMG_WIDTH, IMG_HEIGHT, bbox)\n        bboxes.append(bbox)\n        \n    # Create .txt files for label\n    if row.split == 'train':\n        file_name = f\"tmp/barrier_reef/labels/{row.split}/{row.video_frame}.txt\"\n        os.makedirs(os.path.dirname(file_name), exist_ok=True)\n    else:\n        file_name = f\"tmp/barrier_reef/labels/{row.split}/{row.video_frame}.txt\"\n        os.makedirs(os.path.dirname(file_name), exist_ok=True)\n        \n    # Write the label as .txt file\n    with open(file_name, 'w') as f:\n        for i, bbox in enumerate(bboxes):\n            label = 0\n            bbox = [label]+bbox\n            bbox = [str(i) for i in bbox]\n            bbox = ' '.join(bbox)\n            f.write(bbox)\n            f.write('\\n')","metadata":{"execution":{"iopub.status.busy":"2021-11-23T22:43:47.157803Z","iopub.execute_input":"2021-11-23T22:43:47.158501Z","iopub.status.idle":"2021-11-23T22:43:50.444243Z","shell.execute_reply.started":"2021-11-23T22:43:47.158458Z","shell.execute_reply":"2021-11-23T22:43:50.443363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🚅 Train with W&B","metadata":{}},{"cell_type":"code","source":"%cd tmp/yolov5/","metadata":{"execution":{"iopub.status.busy":"2021-11-23T22:43:50.448318Z","iopub.execute_input":"2021-11-23T22:43:50.450689Z","iopub.status.idle":"2021-11-23T22:43:50.464719Z","shell.execute_reply.started":"2021-11-23T22:43:50.450645Z","shell.execute_reply":"2021-11-23T22:43:50.460598Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"```\n--img {IMG_SIZE} \\ # Input image size.\n--batch {BATCH_SIZE} \\ # Batch size\n--epochs {EPOCHS} \\ # Number of epochs\n--data data.yaml \\ # Configuration file\n--weights yolov5s.pt \\ # Model name\n--save_period 1\\ # Save model after interval\n--project barrier-reef-yolo # W&B project name\n```","metadata":{}},{"cell_type":"code","source":"!python train.py --img 768\\\n                 --batch 16\\\n                 --epochs 10\\\n                 --data data.yaml\\\n                 --weights yolov5s.pt\\\n                 --project barrier-reef-yolo","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# WORK IN PROGRESS","metadata":{}}]}