{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# The Great Barrier Reef:","metadata":{}},{"cell_type":"markdown","source":"![](https://www.worldatlas.com/r/w960-q80/upload/18/9a/e0/shutterstock-642116419.jpg)","metadata":{}},{"cell_type":"markdown","source":"**The Great Barrier Reef is the world’s largest coral reef system located in the Coral Sea off the shore of Queensland, northeastern Australia. It extends over a vast area of approximately 344,4000 sq. km and is is composed of over 2,900 individual reef systems, 760 fringe reefs, 300 coral rays and 900 islands that stretch over 2,300 km.**\n\n**The reef system is composed and built by billions of coral polyps and now supports a wide variety of marine life ranging from ancient sea turtles, reef fish and 134 species of sharks, 400 different hard and soft corals in addition to a plethora of seaweeds. The reef has also cultural significance, as it was used by the Aboriginal Australians and Torres Strait Islanders.**\n\n\n**The Great Barrier Reef is a biodiversity hotspot, housing at least 450 species of hard coral aa well as anemones, sponges, worms, gastropods, lobsters, crayfish, prawns and crabs. More than 1,500 species of fish inhabit the reef, such as wrasses, damselfish, triggerfish, angelfish, rays and sharks. Thirty species of crustaceans have been recorded in the reef, such as the dwarf minke whale, Indo-Pacific humpback dolphin and the humpback whale.Fifteen species of seagrass in beds attract the dugongs and turtles, with six species of sea turtles choosing the reef as its breeding spot. These species include the green sea turtle, leatherback sea turtle, hawksbill turtle, loggerhead sea turtle, flatback turtle and the Olive Ridley.**\n\nsrc: https://www.worldatlas.com/heritage-sites/great-barrier-reef.html","metadata":{}},{"cell_type":"markdown","source":"![](https://www.worldatlas.com/r/w960-q80/upload/39/f6/5d/great-barrier-reef-01.png)","metadata":{}},{"cell_type":"markdown","source":"# Crown-of-Thorns Starfish\n   ","metadata":{}},{"cell_type":"markdown","source":"**In normal numbers on healthy coral reefs, COTS are an important part of the ecosystem. They tend to eat the faster growing corals which gives the slower growing species a chance to catch up, enhancing the coral diversity of our reefs. However, when the coral-eating starfish appear in outbreak proportions, the impact on coral reefs can be disastrous.**\n\n![](https://th.bing.com/th/id/OIP.Et2ArvP-eXTAKdkIp7Vc0QHaFj?w=261&h=195&c=7&r=0&o=5&dpr=1.67&pid=1.7)\n\n**The crown-of-thorns starfish preys on coral polyps. Large outbreaks of these starfish can devastate reefs. In 2000, an outbreak contributed to a loss of 66% of live coral cover on sampled reefs in a study by the Reef Research Centre (RRC). Outbreaks are believed to occur in natural cycles, worsened by poor water quality and overfishing of the starfish's predators.**","metadata":{}},{"cell_type":"markdown","source":"# References and Resources \n* https://www.kaggle.com/remekkinas/sahi-slicing-aided-hyper-inference-yv5-and-yx\n* https://www.kaggle.com/awsaf49/great-barrier-reef-yolov5-train\n* https://www.kaggle.com/remekkinas/yolox-training-pipeline-cots-dataset-lb-0-507?scriptVersionId=81353936\n* https://www.kaggle.com/julian3833/reef-a-cv-strategy-subsequences\n* https://www.kaggle.com/andradaolteanu/greatbarrierreef-full-guide-to-bboxaugmentation\n* https://www.kaggle.com/dschettler8845/tf-find-the-cots-eda-baseline/notebook\n* https://www.kaggle.com/steamedsheep/yolov5-high-resolution-training\n* https://www.kaggle.com/remekkinas/yolox-training-pipeline-cots-dataset-lb-0-507?scriptVersionId=81353936","metadata":{}},{"cell_type":"markdown","source":"# Imports","metadata":{}},{"cell_type":"code","source":"%autosave 200\nimport folium\nimport os \nimport numpy \nimport pandas as pd \nimport matplotlib.pyplot as plt \nimport shutil\nimport numpy as np\n\nfrom tqdm import tqdm","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:23:34.759459Z","iopub.execute_input":"2022-02-13T08:23:34.760322Z","iopub.status.idle":"2022-02-13T08:23:35.087224Z","shell.execute_reply.started":"2022-02-13T08:23:34.760233Z","shell.execute_reply":"2022-02-13T08:23:35.086452Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Weights and Bias Login**","metadata":{}},{"cell_type":"code","source":"import wandb\n\ntry:\n    from kaggle_secrets import UserSecretsClient\n    user_secrets = UserSecretsClient()\n    api_key = user_secrets.get_secret(\"wandb_token\")     # get token from kaggle secrets \n    wandb.login(key=api_key,anonymous=None)                                    # authenticate W and B account\n    \nexcept:\n    wandb.login(anonymous='must')\n    print('To use your W&B account,\\nGo to Add-ons -> Secrets and provide your W&B access token. Use the Label name as WANDB. \\nGet your W&B access token from here: https://wandb.ai/authorize')","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:23:35.088963Z","iopub.execute_input":"2022-02-13T08:23:35.089206Z","iopub.status.idle":"2022-02-13T08:23:36.879472Z","shell.execute_reply.started":"2022-02-13T08:23:35.089173Z","shell.execute_reply":"2022-02-13T08:23:36.878718Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Loading Data","metadata":{}},{"cell_type":"code","source":"train_dir = '../input/tensorflow-great-barrier-reef/train_images' # training data  directory ","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:23:36.882189Z","iopub.execute_input":"2022-02-13T08:23:36.882465Z","iopub.status.idle":"2022-02-13T08:23:36.888431Z","shell.execute_reply.started":"2022-02-13T08:23:36.882430Z","shell.execute_reply":"2022-02-13T08:23:36.887695Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train = pd.read_csv('../input/tensorflow-great-barrier-reef/train.csv')\ntest = pd.read_csv('../input/tensorflow-great-barrier-reef/test.csv')\n\nsample_sub = pd.read_csv('../input/tensorflow-great-barrier-reef/example_sample_submission.csv')\n\n#train\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:23:36.890430Z","iopub.execute_input":"2022-02-13T08:23:36.890812Z","iopub.status.idle":"2022-02-13T08:23:36.965840Z","shell.execute_reply.started":"2022-02-13T08:23:36.890777Z","shell.execute_reply":"2022-02-13T08:23:36.965169Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Basic EDA**","metadata":{}},{"cell_type":"code","source":"#basic eda \nprint('Number of Videos in train data are :',train.video_id.nunique())\n\nfor idx,grp in train.groupby('video_id'):\n    annot= len(grp[grp.annotations!='[]'])\n    print(f'Video {idx} has {len(grp)} Images, of which {annot} Images have Annotations')\n\n\n    \n# marking df rows withoout annotations\n\ntrain.loc[train['annotations']=='[]', 'no_annot'] = 1 \ntrain.loc[train['no_annot']!=1, 'no_annot'] = 0 ","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:23:36.968283Z","iopub.execute_input":"2022-02-13T08:23:36.968550Z","iopub.status.idle":"2022-02-13T08:23:36.997630Z","shell.execute_reply.started":"2022-02-13T08:23:36.968516Z","shell.execute_reply":"2022-02-13T08:23:36.996926Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f'Number of Images without annotations are {train.no_annot.sum()}')\nprint(f'% of Images without annotations are {(train.no_annot.sum()/len(train)).round(2)}')\n","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:23:37.000163Z","iopub.execute_input":"2022-02-13T08:23:37.000348Z","iopub.status.idle":"2022-02-13T08:23:37.005421Z","shell.execute_reply.started":"2022-02-13T08:23:37.000323Z","shell.execute_reply":"2022-02-13T08:23:37.004543Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#checking some annotataions\ntrain[train['no_annot'] != 1].head()","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:23:37.006916Z","iopub.execute_input":"2022-02-13T08:23:37.007223Z","iopub.status.idle":"2022-02-13T08:23:37.025322Z","shell.execute_reply.started":"2022-02-13T08:23:37.007187Z","shell.execute_reply":"2022-02-13T08:23:37.024499Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Taking the data with annotated images**","metadata":{}},{"cell_type":"code","source":"#only taking images with Annotations for training \ntrain_set = train.query(expr = 'no_annot!=1') #drop im without annot \n\ntrain_set.shape","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:23:37.026808Z","iopub.execute_input":"2022-02-13T08:23:37.027057Z","iopub.status.idle":"2022-02-13T08:23:37.039509Z","shell.execute_reply.started":"2022-02-13T08:23:37.027023Z","shell.execute_reply":"2022-02-13T08:23:37.038628Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# YOLO Training","metadata":{}},{"cell_type":"markdown","source":"**Making Directories and copying files to those directories for training.**","metadata":{}},{"cell_type":"code","source":"#directories to copy files \nyolo_train = './yolo_data/fold1/images/train'\nyolo_val   = './yolo_data/fold1/images/val'\n\nyolo_train_labels = './yolo_data/fold1/labels/train'\nyolo_val_labels   =  './yolo_data/fold1/labels/val'","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:23:37.041436Z","iopub.execute_input":"2022-02-13T08:23:37.041735Z","iopub.status.idle":"2022-02-13T08:23:37.046971Z","shell.execute_reply.started":"2022-02-13T08:23:37.041682Z","shell.execute_reply":"2022-02-13T08:23:37.046050Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!mkdir -p $yolo_train\n!mkdir -p $yolo_val\n\n!mkdir -p $yolo_train_labels\n!mkdir -p $yolo_val_labels","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:23:37.048114Z","iopub.execute_input":"2022-02-13T08:23:37.048939Z","iopub.status.idle":"2022-02-13T08:23:39.783843Z","shell.execute_reply.started":"2022-02-13T08:23:37.048900Z","shell.execute_reply":"2022-02-13T08:23:39.782900Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Functions to Copy Images and Convert the Annotations in the format YOLO expects.**","metadata":{}},{"cell_type":"code","source":"def copy_file(filepath,destination):\n    '''copy files from source to dest'''\n    shutil.copy(src=filepath,\n               dst = destination)\n    \n    \ndef get_annotations(annotations,\n                   image_height = 720,\n                   image_width = 1280):\n    '''return annotations formatted in YOLO format ,i.e [x-mid,y-mid,hieght,width], normalized by image height and width'''\n    \n    \n    x_mid = annotations['x'] + annotations['width'] /2\n    y_mid = annotations['y'] + annotations['height'] /2\n    \n    x_mid =x_mid/image_width\n    y_mid=y_mid/image_height\n    \n    width = annotations['width']/image_width\n    height = annotations['height']/image_height\n    \n    return f'0 {x_mid} {y_mid} {width} {height} \\n'\n\ndef save_annot(path_to_save,\n               annot):\n    '''save annotations'''\n    with open(path_to_save,'w') as f:\n        f.write(annot)\n        ","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:23:39.787254Z","iopub.execute_input":"2022-02-13T08:23:39.787496Z","iopub.status.idle":"2022-02-13T08:23:39.797508Z","shell.execute_reply.started":"2022-02-13T08:23:39.787468Z","shell.execute_reply":"2022-02-13T08:23:39.796699Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Moving the data in directories and converting the annotations in YOLO format**","metadata":{}},{"cell_type":"code","source":"#using 2nd video as validation fold :\nyolo_ims = './yolo_data/fold1/images/'\nyolo_lbl = './yolo_data/fold1/labels/'\n\nuse_this_video_as_val = 2 \n\nfor _,row in tqdm(iterable=train_set.iterrows(),\n                  total=len(train_set),\n                  desc = 'Files Copied % :'):        #copy files,convert and save annotations\n    \n    if row.video_id == use_this_video_as_val:\n        set_type = 'val'\n    else:\n        set_type = 'train'\n        \n    \n    video_id = row.video_id\n    video_frame = row.video_frame\n    img_id =  row.image_id\n    annot = eval(row.annotations)[0]    # eval the annotations (which are in format str([x,y,width,height]))\n    \n    \n    file_path = train_dir + f'/video_{video_id}/{video_frame}.jpg'  # filepath of img\n    destination_path = yolo_ims + f'{set_type}/{img_id}.jpg'  #path to copy to \n    \n    copy_file(file_path,destination_path) #copy \n    \n    #get annot in yolo expected format\n    yolo_annot = get_annotations(annot)\n    \n    #save annotations \n    save_annot(path_to_save = yolo_lbl + f'{set_type}/{img_id}.txt' ,\n               annot=yolo_annot)\n    \n    \n    \n","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:23:39.800756Z","iopub.execute_input":"2022-02-13T08:23:39.800979Z","iopub.status.idle":"2022-02-13T08:24:42.311187Z","shell.execute_reply.started":"2022-02-13T08:23:39.800955Z","shell.execute_reply":"2022-02-13T08:24:42.310463Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Cloning YOLO repo","metadata":{}},{"cell_type":"code","source":"!git clone https://github.com/ultralytics/yolov5.git -q","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:24:42.316168Z","iopub.execute_input":"2022-02-13T08:24:42.318243Z","iopub.status.idle":"2022-02-13T08:24:46.879809Z","shell.execute_reply.started":"2022-02-13T08:24:42.318205Z","shell.execute_reply":"2022-02-13T08:24:46.878877Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Saving Hyperparm and Config file for yolo","metadata":{}},{"cell_type":"markdown","source":"**HyperParameters for YOLO:**\n\n\nfrom : from https://www.kaggle.com/steamedsheep/yolov5-high-resolution-training/notebook","metadata":{}},{"cell_type":"code","source":"%%writefile ./yolov5/data/hyps/hyp.heavy.2.yaml\n\n\n# YOLOv5 by Ultralytics, GPL-3.0 license\n# Hyperparameters for COCO training from scratch\n# python train.py --batch 40 --cfg yolov5m.yaml --weights '' --data coco.yaml --img 640 --epochs 300\n# See tutorials for hyperparameter evolution https://github.com/ultralytics/yolov5#tutorials\n\nlr0: 0.01  # initial learning rate (SGD=1E-2, Adam=1E-3)\nlrf: 0.1  # final OneCycleLR learning rate (lr0 * lrf)\nmomentum: 0.937  # SGD momentum/Adam beta1\nweight_decay: 0.0005  # optimizer weight decay 5e-4\nwarmup_epochs: 2.0  # warmup epochs (fractions ok)   (changed from inital 3)\nwarmup_momentum: 0.8  # warmup initial momentum\nwarmup_bias_lr: 0.1  # warmup initial bias lr\nbox: 0.05  # box loss gain\ncls: 0.5  # cls loss gain\ncls_pw: 1.0  # cls BCELoss positive_weight\nobj: 1.0  # obj loss gain (scale with pixels)\nobj_pw: 1.0  # obj BCELoss positive_weight\niou_t: 0.20  # IoU training threshold\nanchor_t: 4.0  # anchor-multiple threshold\n# anchors: 3  # anchors per output layer (0 to ignore)\nfl_gamma: 0.0  # focal loss gamma (efficientDet default gamma=1.5)\nhsv_h: 0.015  # image HSV-Hue augmentation (fraction)\nhsv_s: 0.7  # image HSV-Saturation augmentation (fraction)\nhsv_v: 0.4  # image HSV-Value augmentation (fraction)\ndegrees: 0.0  # image rotation (+/- deg)\ntranslate: 0.1  # image translation (+/- fraction)\nscale: 0.5  # image scale (+/- gain)\nshear: 0.0  # image shear (+/- deg)\nperspective: 0.0  # image perspective (+/- fraction), range 0-0.001\nflipud: 0.5  # image flip up-down (probability)\nfliplr: 0.5  # image flip left-right (probability)\nmosaic: 1.0  # image mosaic (probability)\nmixup: 0.5  # image mixup (probability)\ncopy_paste: 0.0  # segment copy-paste (probability)","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:24:46.883612Z","iopub.execute_input":"2022-02-13T08:24:46.883860Z","iopub.status.idle":"2022-02-13T08:24:46.894087Z","shell.execute_reply.started":"2022-02-13T08:24:46.883831Z","shell.execute_reply":"2022-02-13T08:24:46.893219Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"****","metadata":{}},{"cell_type":"code","source":"%%writefile ./yolov5/data/reef_f1_naive.yaml\n\n# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]\npath: ../yolo_data/fold1/  # dataset root dir\ntrain: images/train  # train images (relative to 'path') 128 images\nval: images/val  # val images (relative to 'path') 128 images\ntest:  # test images (optional)\n\n# Classes\nnc: 1  # number of classes\nnames: ['reef']  # class names\n\n\n# Download script/URL (optional)\n# download: https://ultralytics.com/assets/coco128.zip\n","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:24:46.896678Z","iopub.execute_input":"2022-02-13T08:24:46.897224Z","iopub.status.idle":"2022-02-13T08:24:46.903914Z","shell.execute_reply.started":"2022-02-13T08:24:46.897182Z","shell.execute_reply":"2022-02-13T08:24:46.903184Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training","metadata":{}},{"cell_type":"code","source":"#change current working dir to yolo \n\n%cd yolov5/","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:24:46.905461Z","iopub.execute_input":"2022-02-13T08:24:46.906038Z","iopub.status.idle":"2022-02-13T08:24:46.915974Z","shell.execute_reply.started":"2022-02-13T08:24:46.906001Z","shell.execute_reply":"2022-02-13T08:24:46.915315Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n!python train.py --img 3000 --batch 4 --epochs 8 --data reef_f1_naive.yaml --weights yolov5s6.pt --name l6_3600_uflip_vm5_f1 --hyp data/hyps/hyp.heavy.2.yaml","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:24:46.917314Z","iopub.execute_input":"2022-02-13T08:24:46.917805Z","iopub.status.idle":"2022-02-13T08:29:22.965855Z","shell.execute_reply.started":"2022-02-13T08:24:46.917764Z","shell.execute_reply":"2022-02-13T08:29:22.964672Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Delete Copied files after training**","metadata":{}},{"cell_type":"code","source":"\n!rm -r ../yolo_data/","metadata":{"execution":{"iopub.status.busy":"2022-02-13T08:31:52.716495Z","iopub.execute_input":"2022-02-13T08:31:52.717210Z","iopub.status.idle":"2022-02-13T08:31:53.993643Z","shell.execute_reply.started":"2022-02-13T08:31:52.717170Z","shell.execute_reply":"2022-02-13T08:31:53.992607Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls","metadata":{},"execution_count":null,"outputs":[]}]}