{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"This notebook is a compilation of all object detection models and their training methods using various datasets. The notebook is in making and will be completed by the 13th June 2021","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19"}},{"cell_type":"markdown","source":"Log\n-----------\n\n\n| Version  | Comments  | Status |\n|---|---| ---|\n|  v5 | yolov5 fixed | Done  |\n| v7  | yolov3 added | Done  |\n| v8  | yolov4   | Done |\n| v9  | code fixes  | Done |\n| v10 | ssd_resnet50_v1_fpn_640x640_coco17_tpu-8 | Done|\n| v11 | Fixing the code | Done|\n| v12 | EffDet0 | Due | ","metadata":{}},{"cell_type":"markdown","source":"# Table of Contents\n\n### Let's take a brief overview\n![overview](https://cdn.abcotvs.com/dip/images/208049_072214-cc-mama-giraffe-img.jpg?w=1280&r=16%3A9)\n\n<br><br><br>\n* Theory\n    * Understanding Gradient Vectors, HOG, SS\n    * CNNs, ResNets ,Overfeats\n    * R-CNNs\n    * Fast Detection Models\n* Implementation\n    * YOLO Models\n        * yolov3\n        * Yolov4\n        * yolov4-tiny\n        * yolov4-scaled (csp)\n        * yolov5\n    * Tensorflow2 Object Detection API Models\n        * EfficientDet0\n        * EfficientDet4\n        * CenterNet50 V2 512x512\n        * CenterNet MobileNetV2 FPN 512x512\n        * Mask RCNN\n        * Fast RCNN\n        * Faster RCNN\n        * MobileNet v1 SSD\n        * MobileNet v2 SSD\n    \n    * Detectron2 ","metadata":{}},{"cell_type":"markdown","source":"#### Dataset Gathering for Yolov3 & Yolov5","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Using Blood Cells Detection dataset","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from kaggle_secrets import UserSecretsClient\nuser_secrets = UserSecretsClient()\nsecret_value_0 = user_secrets.get_secret(\"bloodcell_dataset\")\nsecret_value_1 = user_secrets.get_secret(\"bloodcell_dataset_darknet\")\n\n!curl -L -q https://public.roboflow.com/ds/$secret_value_0 > roboflow.zip; unzip roboflow.zip; rm roboflow.zip\n\nimport yaml\nwith open(\"data.yaml\", 'r') as stream:\n    num_classes = str(yaml.safe_load(stream)['nc'])\n    \n    \nnum_classes","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-06-12T18:11:37.176474Z","iopub.execute_input":"2021-06-12T18:11:37.176946Z","iopub.status.idle":"2021-06-12T18:11:41.758599Z","shell.execute_reply.started":"2021-06-12T18:11:37.176848Z","shell.execute_reply":"2021-06-12T18:11:41.757221Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n\n#customize iPython writefile so we can write variables\nfrom IPython.core.magic import register_line_cell_magic\n\n@register_line_cell_magic\ndef writetemplate(line, cell):\n    with open(line, 'w') as f:\n        f.write(cell.format(**globals()))","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:11:41.764013Z","iopub.execute_input":"2021-06-12T18:11:41.766757Z","iopub.status.idle":"2021-06-12T18:11:41.77599Z","shell.execute_reply.started":"2021-06-12T18:11:41.766708Z","shell.execute_reply":"2021-06-12T18:11:41.774557Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"### Yolov3","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!git clone https://github.com/ultralytics/yolov3  # master branch (default)\n    \n%cd yolov3\n","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:11:41.781705Z","iopub.execute_input":"2021-06-12T18:11:41.784927Z","iopub.status.idle":"2021-06-12T18:11:44.974397Z","shell.execute_reply.started":"2021-06-12T18:11:41.78488Z","shell.execute_reply":"2021-06-12T18:11:44.973102Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%writefile data.yaml\n\ntrain: /kaggle/working/train/images\nval: /kaggle/working/valid/images\n\nnc: 3\nnames: ['Platelets', 'RBC', 'WBC']","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:11:44.976951Z","iopub.execute_input":"2021-06-12T18:11:44.977443Z","iopub.status.idle":"2021-06-12T18:11:44.987321Z","shell.execute_reply.started":"2021-06-12T18:11:44.977396Z","shell.execute_reply":"2021-06-12T18:11:44.985938Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install -q -r requirements.txt ","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:11:44.989376Z","iopub.execute_input":"2021-06-12T18:11:44.989983Z","iopub.status.idle":"2021-06-12T18:12:03.309621Z","shell.execute_reply.started":"2021-06-12T18:11:44.989936Z","shell.execute_reply":"2021-06-12T18:12:03.308205Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!python detect.py --source data/images --weights yolov3.pt --conf 0.25","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:12:03.311822Z","iopub.execute_input":"2021-06-12T18:12:03.312352Z","iopub.status.idle":"2021-06-12T18:12:21.131662Z","shell.execute_reply.started":"2021-06-12T18:12:03.31229Z","shell.execute_reply":"2021-06-12T18:12:21.130406Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from PIL import Image\n\nimg1 = Image.open('runs/detect/exp/zidane.jpg')\nimg1","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:12:21.135556Z","iopub.execute_input":"2021-06-12T18:12:21.135901Z","iopub.status.idle":"2021-06-12T18:12:21.690729Z","shell.execute_reply.started":"2021-06-12T18:12:21.135868Z","shell.execute_reply":"2021-06-12T18:12:21.689481Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img2 = Image.open('runs/detect/exp/bus.jpg')\nimg2","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:12:21.694152Z","iopub.execute_input":"2021-06-12T18:12:21.694537Z","iopub.status.idle":"2021-06-12T18:12:22.134667Z","shell.execute_reply.started":"2021-06-12T18:12:21.694496Z","shell.execute_reply":"2021-06-12T18:12:22.133302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls weights","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:12:22.137188Z","iopub.execute_input":"2021-06-12T18:12:22.137677Z","iopub.status.idle":"2021-06-12T18:12:22.943814Z","shell.execute_reply.started":"2021-06-12T18:12:22.13762Z","shell.execute_reply":"2021-06-12T18:12:22.942552Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!wandb offline","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:12:22.946Z","iopub.execute_input":"2021-06-12T18:12:22.946511Z","iopub.status.idle":"2021-06-12T18:12:25.506788Z","shell.execute_reply.started":"2021-06-12T18:12:22.946447Z","shell.execute_reply":"2021-06-12T18:12:25.505256Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%writetemplate ./models/custom_yolov3.yaml\n\n# parameters\nnc: {num_classes}   # number of classes\ndepth_multiple: 0.33  # model depth multiple\nwidth_multiple: 0.50\n\n# anchors\nanchors:\n  - [10,13, 16,30, 33,23]  # P3/8\n  - [30,61, 62,45, 59,119]  # P4/16\n  - [116,90, 156,198, 373,326]  # P5/32\n\n# darknet53 backbone\nbackbone:\n  # [from, number, module, args]\n  [[-1, 1, Conv, [32, 3, 1]],  # 0\n   [-1, 1, Conv, [64, 3, 2]],  # 1-P1/2\n   [-1, 1, Bottleneck, [64]],\n   [-1, 1, Conv, [128, 3, 2]],  # 3-P2/4\n   [-1, 2, Bottleneck, [128]],\n   [-1, 1, Conv, [256, 3, 2]],  # 5-P3/8\n   [-1, 8, Bottleneck, [256]],\n   [-1, 1, Conv, [512, 3, 2]],  # 7-P4/16\n   [-1, 8, Bottleneck, [512]],\n   [-1, 1, Conv, [1024, 3, 2]],  # 9-P5/32\n   [-1, 4, Bottleneck, [1024]],  # 10\n  ]\n\n# YOLOv3 head\nhead:\n  [[-1, 1, Bottleneck, [1024, False]],\n   [-1, 1, Conv, [512, [1, 1]]],\n   [-1, 1, Conv, [1024, 3, 1]],\n   [-1, 1, Conv, [512, 1, 1]],\n   [-1, 1, Conv, [1024, 3, 1]],  # 15 (P5/32-large)\n\n   [-2, 1, Conv, [256, 1, 1]],\n   [-1, 1, nn.Upsample, [None, 2, 'nearest']],\n   [[-1, 8], 1, Concat, [1]],  # cat backbone P4\n   [-1, 1, Bottleneck, [512, False]],\n   [-1, 1, Bottleneck, [512, False]],\n   [-1, 1, Conv, [256, 1, 1]],\n   [-1, 1, Conv, [512, 3, 1]],  # 22 (P4/16-medium)\n\n   [-2, 1, Conv, [128, 1, 1]],\n   [-1, 1, nn.Upsample, [None, 2, 'nearest']],\n   [[-1, 6], 1, Concat, [1]],  # cat backbone P3\n   [-1, 1, Bottleneck, [256, False]],\n   [-1, 2, Bottleneck, [256, False]],  # 27 (P3/8-small)\n\n   [[27, 22, 15], 1, Detect, [nc, anchors]],   # Detect(P3, P4, P5)\n  ]","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:12:25.511991Z","iopub.execute_input":"2021-06-12T18:12:25.512575Z","iopub.status.idle":"2021-06-12T18:12:25.521404Z","shell.execute_reply.started":"2021-06-12T18:12:25.512517Z","shell.execute_reply":"2021-06-12T18:12:25.519218Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Obtain the yolov3 & train","metadata":{}},{"cell_type":"code","source":"!python train.py --img 416 --batch 16 --epochs 40 --data /kaggle/working/data.yaml --cfg ./models/custom_yolov3.yaml --weights yolov3.pt --name yolov3_results  --cache","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:12:25.526852Z","iopub.execute_input":"2021-06-12T18:12:25.529584Z","iopub.status.idle":"2021-06-12T18:21:40.392388Z","shell.execute_reply.started":"2021-06-12T18:12:25.529536Z","shell.execute_reply":"2021-06-12T18:21:40.391121Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls models","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:21:40.394345Z","iopub.execute_input":"2021-06-12T18:21:40.394796Z","iopub.status.idle":"2021-06-12T18:21:41.180388Z","shell.execute_reply.started":"2021-06-12T18:21:40.39475Z","shell.execute_reply":"2021-06-12T18:21:41.179217Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Yolov5","metadata":{}},{"cell_type":"code","source":"%cd ..\n!git clone https://github.com/ultralytics/yolov5","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:21:41.182419Z","iopub.execute_input":"2021-06-12T18:21:41.18288Z","iopub.status.idle":"2021-06-12T18:21:43.990574Z","shell.execute_reply.started":"2021-06-12T18:21:41.182832Z","shell.execute_reply":"2021-06-12T18:21:43.98934Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%cd yolov5","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:21:43.992671Z","iopub.execute_input":"2021-06-12T18:21:43.99318Z","iopub.status.idle":"2021-06-12T18:21:44.001515Z","shell.execute_reply.started":"2021-06-12T18:21:43.99313Z","shell.execute_reply":"2021-06-12T18:21:43.999964Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install -q -U -r requirements.txt  # install dependencies\n\nimport torch\nfrom IPython.display import Image  # for displaying images\nfrom utils.google_utils import gdrive_download  # for downloading models/datasets\nprint('torch %s %s' % (torch.__version__, torch.cuda.get_device_properties(0) if torch.cuda.is_available() else 'CPU'))\n","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-06-12T18:21:44.003153Z","iopub.execute_input":"2021-06-12T18:21:44.003796Z","iopub.status.idle":"2021-06-12T18:24:06.402437Z","shell.execute_reply.started":"2021-06-12T18:21:44.003702Z","shell.execute_reply":"2021-06-12T18:24:06.39948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!python detect.py --weights yolov5x.pt --img 640 --conf 0.25 --source data/images/\n","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:24:06.409001Z","iopub.execute_input":"2021-06-12T18:24:06.411629Z","iopub.status.idle":"2021-06-12T18:24:22.161565Z","shell.execute_reply.started":"2021-06-12T18:24:06.411558Z","shell.execute_reply":"2021-06-12T18:24:22.160203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Image(filename='runs/detect/exp/zidane.jpg', width=600)","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:24:22.165281Z","iopub.execute_input":"2021-06-12T18:24:22.165774Z","iopub.status.idle":"2021-06-12T18:24:22.197445Z","shell.execute_reply.started":"2021-06-12T18:24:22.165725Z","shell.execute_reply":"2021-06-12T18:24:22.196028Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Image(filename='runs/detect/exp/bus.jpg', width=600)","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:24:22.199623Z","iopub.execute_input":"2021-06-12T18:24:22.200451Z","iopub.status.idle":"2021-06-12T18:24:22.231259Z","shell.execute_reply.started":"2021-06-12T18:24:22.200411Z","shell.execute_reply":"2021-06-12T18:24:22.22985Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"testing on yolov5","metadata":{}},{"cell_type":"code","source":"# torch.hub.download_url_to_file('https://github.com/ultralytics/yolov5/releases/download/v1.0/coco2017val.zip', 'tmp.zip')\n# !unzip -q tmp.zip -d ../ && rm tmp.zip","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:24:22.23283Z","iopub.execute_input":"2021-06-12T18:24:22.233634Z","iopub.status.idle":"2021-06-12T18:24:22.239685Z","shell.execute_reply.started":"2021-06-12T18:24:22.233591Z","shell.execute_reply":"2021-06-12T18:24:22.237232Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !python test.py --weights yolov5x.pt --data coco.yaml --img 640 --iou 0.65","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:24:22.243418Z","iopub.execute_input":"2021-06-12T18:24:22.244167Z","iopub.status.idle":"2021-06-12T18:24:22.252534Z","shell.execute_reply.started":"2021-06-12T18:24:22.244115Z","shell.execute_reply":"2021-06-12T18:24:22.251396Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"training on yolov5","metadata":{}},{"cell_type":"code","source":"","metadata":{"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%writetemplate ./models/custom_yolov5s.yaml\n\n# parameters\nnc: {num_classes}  # number of classes\ndepth_multiple: 0.33  # model depth multiple\nwidth_multiple: 0.50  # layer channel multiple\n\n# anchors\nanchors:\n  - [10,13, 16,30, 33,23]  # P3/8\n  - [30,61, 62,45, 59,119]  # P4/16\n  - [116,90, 156,198, 373,326]  # P5/32\n\n# YOLOv5 backbone\nbackbone:\n  # [from, number, module, args]\n  [[-1, 1, Focus, [64, 3]],  # 0-P1/2\n   [-1, 1, Conv, [128, 3, 2]],  # 1-P2/4\n   [-1, 3, BottleneckCSP, [128]],\n   [-1, 1, Conv, [256, 3, 2]],  # 3-P3/8\n   [-1, 9, BottleneckCSP, [256]],\n   [-1, 1, Conv, [512, 3, 2]],  # 5-P4/16\n   [-1, 9, BottleneckCSP, [512]],\n   [-1, 1, Conv, [1024, 3, 2]],  # 7-P5/32\n   [-1, 1, SPP, [1024, [5, 9, 13]]],\n   [-1, 3, BottleneckCSP, [1024, False]],  # 9\n  ]\n\n# YOLOv5 head\nhead:\n  [[-1, 1, Conv, [512, 1, 1]],\n   [-1, 1, nn.Upsample, [None, 2, 'nearest']],\n   [[-1, 6], 1, Concat, [1]],  # cat backbone P4\n   [-1, 3, BottleneckCSP, [512, False]],  # 13\n\n   [-1, 1, Conv, [256, 1, 1]],\n   [-1, 1, nn.Upsample, [None, 2, 'nearest']],\n   [[-1, 4], 1, Concat, [1]],  # cat backbone P3\n   [-1, 3, BottleneckCSP, [256, False]],  # 17 (P3/8-small)\n\n   [-1, 1, Conv, [256, 3, 2]],\n   [[-1, 14], 1, Concat, [1]],  # cat head P4\n   [-1, 3, BottleneckCSP, [512, False]],  # 20 (P4/16-medium)\n\n   [-1, 1, Conv, [512, 3, 2]],\n   [[-1, 10], 1, Concat, [1]],  # cat head P5\n   [-1, 3, BottleneckCSP, [1024, False]],  # 23 (P5/32-large)\n\n   [[17, 20, 23], 1, Detect, [nc, anchors]],  # Detect(P3, P4, P5)\n  ]","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:24:22.254291Z","iopub.execute_input":"2021-06-12T18:24:22.254998Z","iopub.status.idle":"2021-06-12T18:24:22.266367Z","shell.execute_reply.started":"2021-06-12T18:24:22.254941Z","shell.execute_reply":"2021-06-12T18:24:22.26524Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import wandb\n\n# secret_value_1 = user_secrets.get_secret(\"wandb_login\")\n# wandb.login(key = secret_value_1)","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:24:22.280056Z","iopub.execute_input":"2021-06-12T18:24:22.281267Z","iopub.status.idle":"2021-06-12T18:24:22.288252Z","shell.execute_reply.started":"2021-06-12T18:24:22.28122Z","shell.execute_reply":"2021-06-12T18:24:22.287211Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!wandb offline","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:24:22.294178Z","iopub.execute_input":"2021-06-12T18:24:22.295278Z","iopub.status.idle":"2021-06-12T18:24:24.747183Z","shell.execute_reply.started":"2021-06-12T18:24:22.295232Z","shell.execute_reply":"2021-06-12T18:24:24.745809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%cd /kaggle/working/yolov5/\n\n!python train.py --img 416 --batch 16 --epochs 40 --data /kaggle/working/data.yaml --cfg ./models/custom_yolov5s.yaml --weights yolov5x.pt --name yolov5s_results  --cache","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:24:24.751376Z","iopub.execute_input":"2021-06-12T18:24:24.751733Z","iopub.status.idle":"2021-06-12T18:33:14.80623Z","shell.execute_reply.started":"2021-06-12T18:24:24.751698Z","shell.execute_reply":"2021-06-12T18:33:14.804887Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# torch.hub.download_url_to_file('https://github.com/ultralytics/yolov5/releases/download/v1.0/coco128.zip', 'tmp.zip')\n# !unzip -q tmp.zip -d ../ && rm tmp.zip","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:33:14.810315Z","iopub.execute_input":"2021-06-12T18:33:14.810673Z","iopub.status.idle":"2021-06-12T18:33:14.815184Z","shell.execute_reply.started":"2021-06-12T18:33:14.81064Z","shell.execute_reply":"2021-06-12T18:33:14.8137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# %reload_ext tensorboard\n# %tensorboard --logdir runs/train","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:33:14.817019Z","iopub.execute_input":"2021-06-12T18:33:14.817825Z","iopub.status.idle":"2021-06-12T18:33:14.834568Z","shell.execute_reply.started":"2021-06-12T18:33:14.81778Z","shell.execute_reply":"2021-06-12T18:33:14.833172Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !wandb offline","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:33:14.837691Z","iopub.execute_input":"2021-06-12T18:33:14.838032Z","iopub.status.idle":"2021-06-12T18:33:14.84634Z","shell.execute_reply.started":"2021-06-12T18:33:14.838Z","shell.execute_reply":"2021-06-12T18:33:14.845151Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !python train.py --img 640 --batch 16 --epochs 3 --data coco128.yaml --weights yolov5s.pt --nosave --cache","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:33:14.848163Z","iopub.execute_input":"2021-06-12T18:33:14.849095Z","iopub.status.idle":"2021-06-12T18:33:14.85813Z","shell.execute_reply.started":"2021-06-12T18:33:14.849012Z","shell.execute_reply":"2021-06-12T18:33:14.857001Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# from utils.plots import plot_results \n# plot_results(save_dir='runs/train/exp')  # plot all results*.txt as results.png\n# Image(filename='runs/train/exp/results.png', width=800)","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:33:14.859815Z","iopub.execute_input":"2021-06-12T18:33:14.86084Z","iopub.status.idle":"2021-06-12T18:33:14.869645Z","shell.execute_reply.started":"2021-06-12T18:33:14.860719Z","shell.execute_reply":"2021-06-12T18:33:14.86861Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### yolov4 (Darknet Model)","metadata":{}},{"cell_type":"code","source":"%cd /kaggle/working\n!git clone https://github.com/AlexeyAB/darknet.git","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:33:14.871474Z","iopub.execute_input":"2021-06-12T18:33:14.872399Z","iopub.status.idle":"2021-06-12T18:33:18.452495Z","shell.execute_reply.started":"2021-06-12T18:33:14.872352Z","shell.execute_reply":"2021-06-12T18:33:18.451144Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%cd darknet","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:33:18.454704Z","iopub.execute_input":"2021-06-12T18:33:18.455198Z","iopub.status.idle":"2021-06-12T18:33:18.463457Z","shell.execute_reply.started":"2021-06-12T18:33:18.455137Z","shell.execute_reply":"2021-06-12T18:33:18.462187Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# This cell ensures you have the correct architecture for your respective GPU\n# If you command is not found, look through these GPUs, find the respective\n# GPU and add them to the archTypes dictionary\n\n# Tesla V100\n# ARCH= -gencode arch=compute_70,code=[sm_70,compute_70]\n\n# Tesla K80 \n# ARCH= -gencode arch=compute_37,code=sm_37\n\n# GeForce RTX 2080 Ti, RTX 2080, RTX 2070, Quadro RTX 8000, Quadro RTX 6000, Quadro RTX 5000, Tesla T4, XNOR Tensor Cores\n# ARCH= -gencode arch=compute_75,code=[sm_75,compute_75]\n\n# Jetson XAVIER\n# ARCH= -gencode arch=compute_72,code=[sm_72,compute_72]\n\n# GTX 1080, GTX 1070, GTX 1060, GTX 1050, GTX 1030, Titan Xp, Tesla P40, Tesla P4\n# ARCH= -gencode arch=compute_61,code=sm_61\n\n# GP100/Tesla P100 - DGX-1\n# ARCH= -gencode arch=compute_60,code=sm_60\n\n# For Jetson TX1, Tegra X1, DRIVE CX, DRIVE PX - uncomment:\n# ARCH= -gencode arch=compute_53,code=[sm_53,compute_53]\n\n# For Jetson Tx2 or Drive-PX2 uncomment:\n# ARCH= -gencode arch=compute_62,code=[sm_62,compute_62]\nimport os\nos.environ['GPU_TYPE'] = str(os.popen('nvidia-smi --query-gpu=name --format=csv,noheader').read())\n\ndef getGPUArch(argument):\n  try:\n    argument = argument.strip()\n    # All Colab GPUs\n    archTypes = {\n        \"Tesla V100-SXM2-16GB\": \"-gencode arch=compute_70,code=[sm_70,compute_70]\",\n        \"Tesla K80\": \"-gencode arch=compute_37,code=sm_37\",\n        \"Tesla T4\": \"-gencode arch=compute_75,code=[sm_75,compute_75]\",\n        \"Tesla P40\": \"-gencode arch=compute_61,code=sm_61\",\n        \"Tesla P4\": \"-gencode arch=compute_61,code=sm_61\",\n        \"Tesla P100-PCIE-16GB\": \"-gencode arch=compute_60,code=sm_60\"\n\n      }\n    return archTypes[argument]\n  except KeyError:\n    return \"GPU must be added to GPU Commands\"\nos.environ['ARCH_VALUE'] = getGPUArch(os.environ['GPU_TYPE'])\n\nprint(\"GPU Type: \" + os.environ['GPU_TYPE'])\nprint(\"ARCH Value: \" + os.environ['ARCH_VALUE'])","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:33:18.465226Z","iopub.execute_input":"2021-06-12T18:33:18.465786Z","iopub.status.idle":"2021-06-12T18:33:18.504043Z","shell.execute_reply.started":"2021-06-12T18:33:18.465645Z","shell.execute_reply":"2021-06-12T18:33:18.502335Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!sed -i 's/OPENCV=0/OPENCV=1/g' Makefile\n!sed -i 's/GPU=0/GPU=1/g' Makefile\n!sed -i 's/CUDNN=0/CUDNN=1/g' Makefile\n!sed -i 's/CUDNN_HALF=0/CUDNN_HALF=1/g' Makefile\n!sed -i 's/LIBSO=0/LIBSO=1/' Makefile\n!sed -i \"s/ARCH= -gencode arch=compute_60,code=sm_60/ARCH= ${ARCH_VALUE}/g\" Makefile\n!make &> compile.log","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:33:18.506132Z","iopub.execute_input":"2021-06-12T18:33:18.506617Z","iopub.status.idle":"2021-06-12T18:35:06.831661Z","shell.execute_reply.started":"2021-06-12T18:33:18.506571Z","shell.execute_reply":"2021-06-12T18:35:06.830179Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# yolov4 \n!wget https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.conv.137\n\n# yolov4 tiny\n!wget https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v4_pre/yolov4-tiny.conv.29\n    \n    \n    \n!touch ./data/obj.data\n!touch ./data/obj.names","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:06.836198Z","iopub.execute_input":"2021-06-12T18:35:06.836602Z","iopub.status.idle":"2021-06-12T18:35:14.426614Z","shell.execute_reply.started":"2021-06-12T18:35:06.836569Z","shell.execute_reply":"2021-06-12T18:35:14.425218Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%writefile ./data/obj.names\nPlatelets\nRBC\nWBC","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:14.430777Z","iopub.execute_input":"2021-06-12T18:35:14.431211Z","iopub.status.idle":"2021-06-12T18:35:14.437962Z","shell.execute_reply.started":"2021-06-12T18:35:14.431176Z","shell.execute_reply":"2021-06-12T18:35:14.436779Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"## Make a folder to store the experiment weights and backups\n!mkdir training\n\n# Store weights for the nth experiment\n!mkdir training/experiment1","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:14.439494Z","iopub.execute_input":"2021-06-12T18:35:14.440485Z","iopub.status.idle":"2021-06-12T18:35:15.981444Z","shell.execute_reply.started":"2021-06-12T18:35:14.440441Z","shell.execute_reply":"2021-06-12T18:35:15.98011Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"num_classes = 3","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:15.983587Z","iopub.execute_input":"2021-06-12T18:35:15.984097Z","iopub.status.idle":"2021-06-12T18:35:15.992372Z","shell.execute_reply.started":"2021-06-12T18:35:15.984047Z","shell.execute_reply":"2021-06-12T18:35:15.989531Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%writetemplate ./data/obj.data\nclasses = {num_classes}\ntrain = data/train.txt\nvalid = data/test.txt \nnames = data/obj.names \nbackup = /kaggle/working/darknet/training/experiment1","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:15.994325Z","iopub.execute_input":"2021-06-12T18:35:15.994942Z","iopub.status.idle":"2021-06-12T18:35:16.005339Z","shell.execute_reply.started":"2021-06-12T18:35:15.994897Z","shell.execute_reply":"2021-06-12T18:35:16.004055Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cat ./data/obj.data","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:16.009329Z","iopub.execute_input":"2021-06-12T18:35:16.009685Z","iopub.status.idle":"2021-06-12T18:35:16.786816Z","shell.execute_reply.started":"2021-06-12T18:35:16.009654Z","shell.execute_reply":"2021-06-12T18:35:16.785616Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls cfg","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:16.790617Z","iopub.execute_input":"2021-06-12T18:35:16.790934Z","iopub.status.idle":"2021-06-12T18:35:17.577205Z","shell.execute_reply.started":"2021-06-12T18:35:16.790902Z","shell.execute_reply":"2021-06-12T18:35:17.575923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"## View the default config of yolov4\n\n!cat cfg/yolov4.cfg","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-06-12T18:35:17.581344Z","iopub.execute_input":"2021-06-12T18:35:17.581679Z","iopub.status.idle":"2021-06-12T18:35:18.360982Z","shell.execute_reply.started":"2021-06-12T18:35:17.581644Z","shell.execute_reply":"2021-06-12T18:35:18.35972Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"num_classes = 3\nmax_batches = 2100 #any number of iterations, ideally num_classes * 2000. Due to computation limits, limiting to 2200 iterations\nstep1 = int(0.8*max_batches)\nstep2 = int(0.9*max_batches)\n\nfilters = (num_classes + 5)*3\n\nprint(num_classes)\nprint(max_batches)\nprint(step1)\nprint(step2)","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:18.363109Z","iopub.execute_input":"2021-06-12T18:35:18.363587Z","iopub.status.idle":"2021-06-12T18:35:18.37329Z","shell.execute_reply.started":"2021-06-12T18:35:18.363537Z","shell.execute_reply":"2021-06-12T18:35:18.371901Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%writetemplate cfg/yolov4.cfg\n[net]\nbatch=64\nsubdivisions=16\n# Training\n#width=512\n#height=512\nwidth=608\nheight=608\nchannels=3\nmomentum=0.949\ndecay=0.0005\nangle=0\nsaturation = 1.5\nexposure = 1.5\nhue=.1\n\nlearning_rate=0.0013\nburn_in=1000\nmax_batches = 2200 #{num_classes}*2000\npolicy=steps\nsteps={step1}, {step2}\nscales=.1,.1\n\n#cutmix=1\nmosaic=1\n\n#:104x104 54:52x52 85:26x26 104:13x13 for 416\n\n[convolutional]\nbatch_normalize=1\nfilters=32\nsize=3\nstride=1\npad=1\nactivation=mish\n\n# Downsample\n\n[convolutional]\nbatch_normalize=1\nfilters=64\nsize=3\nstride=2\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=64\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[route]\nlayers = -2\n\n[convolutional]\nbatch_normalize=1\nfilters=64\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=32\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=64\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=64\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[route]\nlayers = -1,-7\n\n[convolutional]\nbatch_normalize=1\nfilters=64\nsize=1\nstride=1\npad=1\nactivation=mish\n\n# Downsample\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=3\nstride=2\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=64\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[route]\nlayers = -2\n\n[convolutional]\nbatch_normalize=1\nfilters=64\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=64\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=64\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=64\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=64\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=64\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[route]\nlayers = -1,-10\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=mish\n\n# Downsample\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=3\nstride=2\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[route]\nlayers = -2\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[route]\nlayers = -1,-28\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=mish\n\n# Downsample\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=3\nstride=2\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[route]\nlayers = -2\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[route]\nlayers = -1,-28\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=1\nstride=1\npad=1\nactivation=mish\n\n# Downsample\n\n[convolutional]\nbatch_normalize=1\nfilters=1024\nsize=3\nstride=2\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[route]\nlayers = -2\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=3\nstride=1\npad=1\nactivation=mish\n\n[shortcut]\nfrom=-3\nactivation=linear\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=1\nstride=1\npad=1\nactivation=mish\n\n[route]\nlayers = -1,-16\n\n[convolutional]\nbatch_normalize=1\nfilters=1024\nsize=1\nstride=1\npad=1\nactivation=mish\n\n##########################\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nsize=3\nstride=1\npad=1\nfilters=1024\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n### SPP ###\n[maxpool]\nstride=1\nsize=5\n\n[route]\nlayers=-2\n\n[maxpool]\nstride=1\nsize=9\n\n[route]\nlayers=-4\n\n[maxpool]\nstride=1\nsize=13\n\n[route]\nlayers=-1,-3,-5,-6\n### End SPP ###\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nsize=3\nstride=1\npad=1\nfilters=1024\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[upsample]\nstride=2\n\n[route]\nlayers = 85\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[route]\nlayers = -1, -3\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nsize=3\nstride=1\npad=1\nfilters=512\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nsize=3\nstride=1\npad=1\nfilters=512\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[upsample]\nstride=2\n\n[route]\nlayers = 54\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[route]\nlayers = -1, -3\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nsize=3\nstride=1\npad=1\nfilters=256\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nsize=3\nstride=1\npad=1\nfilters=256\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nfilters=128\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n##########################\n\n[convolutional]\nbatch_normalize=1\nsize=3\nstride=1\npad=1\nfilters=256\nactivation=leaky\n\n[convolutional]\nsize=1\nstride=1\npad=1\nfilters={filters}\nactivation=linear\n\n\n[yolo]\nmask = 0,1,2\nanchors = 12, 16, 19, 36, 40, 28, 36, 75, 76, 55, 72, 146, 142, 110, 192, 243, 459, 401\nclasses={num_classes}\nnum=9\njitter=.3\nignore_thresh = .7\ntruth_thresh = 1\nscale_x_y = 1.2\niou_thresh=0.213\ncls_normalizer=1.0\niou_normalizer=0.07\niou_loss=ciou\nnms_kind=greedynms\nbeta_nms=0.6\nmax_delta=5\n\n\n[route]\nlayers = -4\n\n[convolutional]\nbatch_normalize=1\nsize=3\nstride=2\npad=1\nfilters=256\nactivation=leaky\n\n[route]\nlayers = -1, -16\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nsize=3\nstride=1\npad=1\nfilters=512\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nsize=3\nstride=1\npad=1\nfilters=512\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nfilters=256\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nsize=3\nstride=1\npad=1\nfilters=512\nactivation=leaky\n\n[convolutional]\nsize=1\nstride=1\npad=1\nfilters={filters}\nactivation=linear\n\n\n[yolo]\nmask = 3,4,5\nanchors = 12, 16, 19, 36, 40, 28, 36, 75, 76, 55, 72, 146, 142, 110, 192, 243, 459, 401\nclasses={num_classes}\nnum=9\njitter=.3\nignore_thresh = .7\ntruth_thresh = 1\nscale_x_y = 1.1\niou_thresh=0.213\ncls_normalizer=1.0\niou_normalizer=0.07\niou_loss=ciou\nnms_kind=greedynms\nbeta_nms=0.6\nmax_delta=5\n\n\n[route]\nlayers = -4\n\n[convolutional]\nbatch_normalize=1\nsize=3\nstride=2\npad=1\nfilters=512\nactivation=leaky\n\n[route]\nlayers = -1, -37\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nsize=3\nstride=1\npad=1\nfilters=1024\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nsize=3\nstride=1\npad=1\nfilters=1024\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nfilters=512\nsize=1\nstride=1\npad=1\nactivation=leaky\n\n[convolutional]\nbatch_normalize=1\nsize=3\nstride=1\npad=1\nfilters=1024\nactivation=leaky\n\n[convolutional]\nsize=1\nstride=1\npad=1\nfilters={filters}\nactivation=linear\n\n\n[yolo]\nmask = 6,7,8\nanchors = 12, 16, 19, 36, 40, 28, 36, 75, 76, 55, 72, 146, 142, 110, 192, 243, 459, 401\nclasses={num_classes}\nnum=9\njitter=.3\nignore_thresh = .7\ntruth_thresh = 1\nrandom=1\nscale_x_y = 1.05\niou_thresh=0.213\ncls_normalizer=1.0\niou_normalizer=0.07\niou_loss=ciou\nnms_kind=greedynms\nbeta_nms=0.6\nmax_delta=5","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:18.375295Z","iopub.execute_input":"2021-06-12T18:35:18.375753Z","iopub.status.idle":"2021-06-12T18:35:18.387325Z","shell.execute_reply.started":"2021-06-12T18:35:18.375705Z","shell.execute_reply":"2021-06-12T18:35:18.385877Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cat cfg/yolov4.cfg","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-06-12T18:35:18.389196Z","iopub.execute_input":"2021-06-12T18:35:18.390178Z","iopub.status.idle":"2021-06-12T18:35:19.17831Z","shell.execute_reply.started":"2021-06-12T18:35:18.390128Z","shell.execute_reply":"2021-06-12T18:35:19.176769Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Prepare dataset in yolo darknet format","metadata":{}},{"cell_type":"code","source":"!ls ./data","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:19.180456Z","iopub.execute_input":"2021-06-12T18:35:19.180966Z","iopub.status.idle":"2021-06-12T18:35:20.02332Z","shell.execute_reply.started":"2021-06-12T18:35:19.180916Z","shell.execute_reply":"2021-06-12T18:35:20.02203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!mkdir ./data/obj","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:20.026827Z","iopub.execute_input":"2021-06-12T18:35:20.033249Z","iopub.status.idle":"2021-06-12T18:35:21.384921Z","shell.execute_reply.started":"2021-06-12T18:35:20.033204Z","shell.execute_reply":"2021-06-12T18:35:21.383389Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from kaggle_secrets import UserSecretsClient\nuser_secrets = UserSecretsClient()\nsecret_value_1 = user_secrets.get_secret(\"bloodcell_dataset_darknet\")\n!curl -L -q https://public.roboflow.com/ds/$secret_value_1 > roboflow.zip; unzip roboflow.zip; rm roboflow.zip\n","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-06-12T18:35:21.390538Z","iopub.execute_input":"2021-06-12T18:35:21.393142Z","iopub.status.idle":"2021-06-12T18:35:25.101655Z","shell.execute_reply.started":"2021-06-12T18:35:21.393061Z","shell.execute_reply":"2021-06-12T18:35:25.100383Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls train | wc -l","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:25.105791Z","iopub.execute_input":"2021-06-12T18:35:25.106186Z","iopub.status.idle":"2021-06-12T18:35:25.895296Z","shell.execute_reply.started":"2021-06-12T18:35:25.106155Z","shell.execute_reply":"2021-06-12T18:35:25.894139Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls test | wc -l ","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:25.898999Z","iopub.execute_input":"2021-06-12T18:35:25.899331Z","iopub.status.idle":"2021-06-12T18:35:26.690082Z","shell.execute_reply.started":"2021-06-12T18:35:25.899299Z","shell.execute_reply":"2021-06-12T18:35:26.688784Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls valid | wc -l","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:26.692217Z","iopub.execute_input":"2021-06-12T18:35:26.692682Z","iopub.status.idle":"2021-06-12T18:35:27.483504Z","shell.execute_reply.started":"2021-06-12T18:35:26.692633Z","shell.execute_reply":"2021-06-12T18:35:27.482148Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#copy all under obj folder","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:27.485496Z","iopub.execute_input":"2021-06-12T18:35:27.486022Z","iopub.status.idle":"2021-06-12T18:35:27.493238Z","shell.execute_reply.started":"2021-06-12T18:35:27.485961Z","shell.execute_reply":"2021-06-12T18:35:27.490496Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cp train/* data/obj/\n!cp test/* data/obj/\n!cp valid/* data/obj/\n\n!touch process.py","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:27.494659Z","iopub.execute_input":"2021-06-12T18:35:27.495005Z","iopub.status.idle":"2021-06-12T18:35:30.779038Z","shell.execute_reply.started":"2021-06-12T18:35:27.494974Z","shell.execute_reply":"2021-06-12T18:35:30.777503Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%writetemplate process.py\nimport glob, os\n\n# Current directory\ncurrent_dir = os.path.dirname(os.path.abspath(__file__))\n\nprint(current_dir)\n\ncurrent_dir = 'data/obj'\n\n# Percentage of images to be used for the test set\npercentage_test = 10;\n\n# Create and/or truncate train.txt and test.txt\nfile_train = open('data/train.txt', 'w')\nfile_test = open('data/test.txt', 'w')\n\n# Populate train.txt and test.txt\ncounter = 1\nindex_test = round(100 / percentage_test)\nfor pathAndFilename in glob.iglob(os.path.join(current_dir, \"*.jpg\")):\n    title, ext = os.path.splitext(os.path.basename(pathAndFilename))\n\n    if counter == index_test:\n        counter = 1\n        file_test.write(\"data/obj\" + \"/\" + title + '.jpg' + \"\\n\")\n    else:\n        file_train.write(\"data/obj\" + \"/\" + title + '.jpg' + \"\\n\")\n        counter = counter + 1","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:30.78327Z","iopub.execute_input":"2021-06-12T18:35:30.783662Z","iopub.status.idle":"2021-06-12T18:35:30.793004Z","shell.execute_reply.started":"2021-06-12T18:35:30.783625Z","shell.execute_reply":"2021-06-12T18:35:30.791862Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!python process.py","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:30.79553Z","iopub.execute_input":"2021-06-12T18:35:30.796034Z","iopub.status.idle":"2021-06-12T18:35:31.812172Z","shell.execute_reply.started":"2021-06-12T18:35:30.79598Z","shell.execute_reply":"2021-06-12T18:35:31.81077Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cat data/train.txt | wc -l","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:31.817678Z","iopub.execute_input":"2021-06-12T18:35:31.820453Z","iopub.status.idle":"2021-06-12T18:35:32.662526Z","shell.execute_reply.started":"2021-06-12T18:35:31.820379Z","shell.execute_reply":"2021-06-12T18:35:32.66131Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cat data/test.txt | wc -l","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:32.664568Z","iopub.execute_input":"2021-06-12T18:35:32.665039Z","iopub.status.idle":"2021-06-12T18:35:33.456063Z","shell.execute_reply.started":"2021-06-12T18:35:32.664988Z","shell.execute_reply":"2021-06-12T18:35:33.4548Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"Scratch training 2000 iterations","metadata":{}},{"cell_type":"code","source":"!./darknet.exe detector train data/obj.data cfg/yolov4.cfg -dont_show -map","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:33.459939Z","iopub.execute_input":"2021-06-12T18:35:33.460308Z","iopub.status.idle":"2021-06-12T18:35:34.231004Z","shell.execute_reply.started":"2021-06-12T18:35:33.460273Z","shell.execute_reply":"2021-06-12T18:35:34.229814Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## And we just ran into an error. ","metadata":{"execution":{"iopub.status.busy":"2021-06-09T02:32:03.64423Z","iopub.execute_input":"2021-06-09T02:32:03.644582Z","iopub.status.idle":"2021-06-09T02:32:04.290525Z","shell.execute_reply.started":"2021-06-09T02:32:03.644545Z","shell.execute_reply":"2021-06-09T02:32:04.289483Z"}}},{"cell_type":"markdown","source":"![](https://i.redd.it/qp020ibgi7v41.jpg)","metadata":{}},{"cell_type":"markdown","source":"### kaggle has permission issues with cuda & cudnn. Follow the procedure on colab or local till a workaround is found for Kaggle Containers. <br><br>","metadata":{}},{"cell_type":"markdown","source":"Pretrained Training 2000 iterations","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#!./darknet detector train data/obj.data cfg/yolov4-tiny.cfg yolov4-tiny.conv.29 -dont_show -map","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:34.234651Z","iopub.execute_input":"2021-06-12T18:35:34.235006Z","iopub.status.idle":"2021-06-12T18:35:34.240891Z","shell.execute_reply.started":"2021-06-12T18:35:34.234975Z","shell.execute_reply":"2021-06-12T18:35:34.239611Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Tensorflow2 Object Detection","metadata":{}},{"cell_type":"markdown","source":"#### This might seem difficult in the beginning, but don't worry and feel left out. \n\n![](https://images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com/f/b2f14935-b7a6-46bb-8640-77626867290b/da9zemv-0411af2e-022c-4e00-b434-a865a0ea6de8.jpg/v1/fill/w_1024,h_851,q_75,strp/diana_the_vampire_rabbit_by_pythos_cheetah_da9zemv-fullview.jpg?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1cm46YXBwOjdlMGQxODg5ODIyNjQzNzNhNWYwZDQxNWVhMGQyNmUwIiwiaXNzIjoidXJuOmFwcDo3ZTBkMTg4OTgyMjY0MzczYTVmMGQ0MTVlYTBkMjZlMCIsIm9iaiI6W1t7ImhlaWdodCI6Ijw9ODUxIiwicGF0aCI6IlwvZlwvYjJmMTQ5MzUtYjdhNi00NmJiLTg2NDAtNzc2MjY4NjcyOTBiXC9kYTl6ZW12LTA0MTFhZjJlLTAyMmMtNGUwMC1iNDM0LWE4NjVhMGVhNmRlOC5qcGciLCJ3aWR0aCI6Ijw9MTAyNCJ9XV0sImF1ZCI6WyJ1cm46c2VydmljZTppbWFnZS5vcGVyYXRpb25zIl19.NcSS0mvUQnDVbDWnTi6dlK49mLVQ0Yx4et4r3E1UbgE)","metadata":{}},{"cell_type":"markdown","source":"### Data Science community is with you! :)\n\n![](https://cdn4.eyeem.com/thumb/a73ebc988569342622aa9e884a419bf433032f81-1539776693848/w/1280)","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##### Brief Overview of the TF2 Object Detection Project Workflow (under making)","metadata":{}},{"cell_type":"markdown","source":"Data Preparation\n\nTFRecord dataset","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"##### Designing your Training Workspace","metadata":{}},{"cell_type":"markdown","source":"TensorFlow/<br>\n├─ addons/ (Optional)<br>\n│  └─ labelImg/<br>\n├─ models/ <br>\n│  ├─ community/ <br>\n│  ├─ official/ <br>\n│  ├─ orbit/<br>\n│  ├─ research/<br>\n│  └─ ...<br>\n├─ scripts/<br>\n│  └─ preprocessing/<br>\n└─ workspace/ <br>\n   └─ training_demo/ <br>","metadata":{}},{"cell_type":"markdown","source":"Here, models is the tensorflow2 models api git clone <br>\nworkspace is where training and loading of models happen","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### ssd_resnet50_v1_fpn_640x640_coco17_tpu-8\n","metadata":{}},{"cell_type":"markdown","source":"Here, in this example, I am using ready-made TFRecords to simplify understanding. We'll take on to creating custom TFRecords dataset soo but not now. That would be information overload.","metadata":{}},{"cell_type":"code","source":"!rm /kaggle/working/README.roboflow.txt\n!rm /kaggle/working/README.dataset.txt","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:34.243083Z","iopub.execute_input":"2021-06-12T18:35:34.243558Z","iopub.status.idle":"2021-06-12T18:35:35.784429Z","shell.execute_reply.started":"2021-06-12T18:35:34.243513Z","shell.execute_reply":"2021-06-12T18:35:35.783045Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from kaggle_secrets import UserSecretsClient\nuser_secrets = UserSecretsClient()\nsecret_value_2 = user_secrets.get_secret(\"bloodcell_dataset_tf_tfrecords\")\n\n#Downloading data from Roboflow\n#UPDATE THIS LINK - get our data from Roboflow\n%cd /kaggle/working\n!curl -L https://public.roboflow.com/ds/$secret_value_2 > roboflow.zip; unzip roboflow.zip; rm roboflow.zip","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:35:35.78738Z","iopub.execute_input":"2021-06-12T18:35:35.788381Z","iopub.status.idle":"2021-06-12T18:35:38.604937Z","shell.execute_reply.started":"2021-06-12T18:35:35.788315Z","shell.execute_reply":"2021-06-12T18:35:38.603629Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:36:33.736676Z","iopub.execute_input":"2021-06-12T18:36:33.737157Z","iopub.status.idle":"2021-06-12T18:36:34.516187Z","shell.execute_reply.started":"2021-06-12T18:36:33.737121Z","shell.execute_reply":"2021-06-12T18:36:34.514944Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!mkdir Tensorflow\n%cd Tensorflow","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:36:43.478238Z","iopub.execute_input":"2021-06-12T18:36:43.478637Z","iopub.status.idle":"2021-06-12T18:36:44.256413Z","shell.execute_reply.started":"2021-06-12T18:36:43.478605Z","shell.execute_reply":"2021-06-12T18:36:44.255181Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!git clone https://github.com/tensorflow/models.git\n%cd /kaggle/working/Tensorflow/models/research\n!protoc ./object_detection/protos/*.proto --python_out=.\n!cp ./object_detection/packages/tf2/setup.py .\n!python -m pip install --user .\n\n\n!pip install tensorflow==2.4.1\n!pip install tensorflow-addons\n\n\n!python ./object_detection/builders/model_builder_tf2_test.py\n\n","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-06-12T18:36:46.200971Z","iopub.execute_input":"2021-06-12T18:36:46.201419Z","iopub.status.idle":"2021-06-12T18:40:35.976793Z","shell.execute_reply.started":"2021-06-12T18:36:46.201384Z","shell.execute_reply":"2021-06-12T18:40:35.975479Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Library & Dependency setup complete! ","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# %cd Tensorflow\n# !git clone https://github.com/tensorflow/models","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:08.980187Z","iopub.status.idle":"2021-06-12T18:02:08.980587Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# %cd ./models/research/\n# !protoc ./object_detection/protos/*.proto --python_out=.","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:08.981712Z","iopub.status.idle":"2021-06-12T18:02:08.982429Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !cat ./object_detection/packages/tf2/setup.py","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:08.98389Z","iopub.status.idle":"2021-06-12T18:02:08.984542Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !protoc object_detection/protos/*.proto --python_out=.","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:08.985649Z","iopub.status.idle":"2021-06-12T18:02:08.986383Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !cp ./object_detection/packages/tf2/setup.py .\n# !python -m pip -q install .","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:08.987456Z","iopub.status.idle":"2021-06-12T18:02:08.98807Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !pip install scipy==1.5.4\n# !pip install dill==0.3.3\n# !pip install pathos==0.2.7\n# !pip install multiprocess==0.70.11.1\n# !pip install autogluon-core==0.1.0","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:08.989334Z","iopub.status.idle":"2021-06-12T18:02:08.989935Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !pip install tf_slim\n\n# !pip install pycocotools\n# !pip install lvis\n# !pip install numba ","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:08.991151Z","iopub.status.idle":"2021-06-12T18:02:08.992053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !python object_detection/builders/model_builder_tf2_test.py","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:08.993141Z","iopub.status.idle":"2021-06-12T18:02:08.993772Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import tensorflow as tf\ntf.__version__","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:40:35.980454Z","iopub.execute_input":"2021-06-12T18:40:35.980947Z","iopub.status.idle":"2021-06-12T18:40:38.01651Z","shell.execute_reply.started":"2021-06-12T18:40:35.980883Z","shell.execute_reply":"2021-06-12T18:40:38.015447Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# %cd /kaggle/working\n\n# !mkdir Tensorflow","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:08.996621Z","iopub.status.idle":"2021-06-12T18:02:08.99722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# %cd /kaggle/working/Tensorflow","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:08.998295Z","iopub.status.idle":"2021-06-12T18:02:08.998931Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##### Generating the desired directory structure","metadata":{}},{"cell_type":"code","source":"!pwd","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:40:38.019043Z","iopub.execute_input":"2021-06-12T18:40:38.019565Z","iopub.status.idle":"2021-06-12T18:40:38.808899Z","shell.execute_reply.started":"2021-06-12T18:40:38.019504Z","shell.execute_reply":"2021-06-12T18:40:38.80775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%cd /kaggle/working/Tensorflow","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:40:38.812988Z","iopub.execute_input":"2021-06-12T18:40:38.813395Z","iopub.status.idle":"2021-06-12T18:40:38.824485Z","shell.execute_reply.started":"2021-06-12T18:40:38.813363Z","shell.execute_reply":"2021-06-12T18:40:38.822637Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!mkdir workspace","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:40:38.827895Z","iopub.execute_input":"2021-06-12T18:40:38.828272Z","iopub.status.idle":"2021-06-12T18:40:39.673527Z","shell.execute_reply.started":"2021-06-12T18:40:38.828242Z","shell.execute_reply":"2021-06-12T18:40:39.672067Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/working/Tensorflow","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:40:39.679216Z","iopub.execute_input":"2021-06-12T18:40:39.681911Z","iopub.status.idle":"2021-06-12T18:40:40.566348Z","shell.execute_reply.started":"2021-06-12T18:40:39.681858Z","shell.execute_reply":"2021-06-12T18:40:40.565029Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!mkdir workspace/training_demo","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:40:40.570384Z","iopub.execute_input":"2021-06-12T18:40:40.570753Z","iopub.status.idle":"2021-06-12T18:40:41.355431Z","shell.execute_reply.started":"2021-06-12T18:40:40.570721Z","shell.execute_reply":"2021-06-12T18:40:41.354038Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/working/Tensorflow/workspace","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:40:41.360558Z","iopub.execute_input":"2021-06-12T18:40:41.360907Z","iopub.status.idle":"2021-06-12T18:40:42.147565Z","shell.execute_reply.started":"2021-06-12T18:40:41.360872Z","shell.execute_reply":"2021-06-12T18:40:42.146333Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!mkdir workspace/training_demo/annotations","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:40:42.149991Z","iopub.execute_input":"2021-06-12T18:40:42.150516Z","iopub.status.idle":"2021-06-12T18:40:42.935403Z","shell.execute_reply.started":"2021-06-12T18:40:42.150464Z","shell.execute_reply":"2021-06-12T18:40:42.933998Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"labelmap.pbtxt should be under training_demo/annotations <br>\ntest.record should be under training_demo/annotations <br>\ntrain.record should be under training_demo/annotations <br>\n","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/working/Tensorflow/","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:40:42.939526Z","iopub.execute_input":"2021-06-12T18:40:42.939907Z","iopub.status.idle":"2021-06-12T18:40:43.723153Z","shell.execute_reply.started":"2021-06-12T18:40:42.939874Z","shell.execute_reply":"2021-06-12T18:40:43.721987Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/working/train\n\n!cp -r /kaggle/working/train/* /kaggle/working/Tensorflow/workspace/training_demo/annotations/\n!mv /kaggle/working/Tensorflow/workspace/training_demo/annotations/cells.tfrecord /kaggle/working/Tensorflow/workspace/training_demo/annotations/cells_train.tfrecord\n!cp -r /kaggle/working/test/* /kaggle/working/Tensorflow/workspace/training_demo/annotations/\n!mv /kaggle/working/Tensorflow/workspace/training_demo/annotations/cells.tfrecord /kaggle/working/Tensorflow/workspace/training_demo/annotations/cells_test.tfrecord","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:40:43.725333Z","iopub.execute_input":"2021-06-12T18:40:43.725804Z","iopub.status.idle":"2021-06-12T18:40:48.097715Z","shell.execute_reply.started":"2021-06-12T18:40:43.725756Z","shell.execute_reply":"2021-06-12T18:40:48.096265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cat /kaggle/working/train/cells_label_map.pbtxt","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:40:49.948321Z","iopub.execute_input":"2021-06-12T18:40:49.948728Z","iopub.status.idle":"2021-06-12T18:40:50.841079Z","shell.execute_reply.started":"2021-06-12T18:40:49.948689Z","shell.execute_reply":"2021-06-12T18:40:50.839748Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**the pretrained model should be downloaded under training_demo/pre-trainined-models**","metadata":{}},{"cell_type":"code","source":"!pwd","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:40:54.314738Z","iopub.execute_input":"2021-06-12T18:40:54.315152Z","iopub.status.idle":"2021-06-12T18:40:55.097767Z","shell.execute_reply.started":"2021-06-12T18:40:54.315114Z","shell.execute_reply":"2021-06-12T18:40:55.096464Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!mkdir /kaggle/working/Tensorflow/workspace/training_demo/pre-trained-models","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:40:58.149986Z","iopub.execute_input":"2021-06-12T18:40:58.15048Z","iopub.status.idle":"2021-06-12T18:40:58.932518Z","shell.execute_reply.started":"2021-06-12T18:40:58.150429Z","shell.execute_reply":"2021-06-12T18:40:58.931164Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%cd /kaggle/working/Tensorflow/workspace/training_demo/pre-trained-models","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:00.211881Z","iopub.execute_input":"2021-06-12T18:41:00.212321Z","iopub.status.idle":"2021-06-12T18:41:00.223538Z","shell.execute_reply.started":"2021-06-12T18:41:00.212283Z","shell.execute_reply":"2021-06-12T18:41:00.22205Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Under pretrained models folder download all the pretrained models you wish to experiment with","metadata":{}},{"cell_type":"code","source":"\n\nMODEL = \"faster_rcnn_resnet50_v1_1024x1024_coco17_tpu-8\"\n\n!wget http://download.tensorflow.org/models/object_detection/tf2/20200711/{MODEL}.tar.gz\n!tar -xf {MODEL}.tar.gz\n!rm {MODEL}.tar.gz","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:04.321779Z","iopub.execute_input":"2021-06-12T18:41:04.322195Z","iopub.status.idle":"2021-06-12T18:41:14.941043Z","shell.execute_reply.started":"2021-06-12T18:41:04.32216Z","shell.execute_reply":"2021-06-12T18:41:14.939646Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/working/Tensorflow/workspace/training_demo/pre-trained-models","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:14.945242Z","iopub.execute_input":"2021-06-12T18:41:14.945603Z","iopub.status.idle":"2021-06-12T18:41:15.730754Z","shell.execute_reply.started":"2021-06-12T18:41:14.945552Z","shell.execute_reply":"2021-06-12T18:41:15.729495Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"MODEL2 = 'efficientdet_d1_coco17_tpu-32'\n\n!wget http://download.tensorflow.org/models/object_detection/tf2/20200711/{MODEL2}.tar.gz\n!tar -xf {MODEL2}.tar.gz\n!rm {MODEL2}.tar.gz","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:15.733505Z","iopub.execute_input":"2021-06-12T18:41:15.733973Z","iopub.status.idle":"2021-06-12T18:41:22.21625Z","shell.execute_reply.started":"2021-06-12T18:41:15.733927Z","shell.execute_reply":"2021-06-12T18:41:22.214852Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/working/Tensorflow/workspace/training_demo/pre-trained-models","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:22.218806Z","iopub.execute_input":"2021-06-12T18:41:22.219308Z","iopub.status.idle":"2021-06-12T18:41:23.004695Z","shell.execute_reply.started":"2021-06-12T18:41:22.219258Z","shell.execute_reply":"2021-06-12T18:41:23.003484Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"MODEL3 = 'ssd_resnet50_v1_fpn_640x640_coco17_tpu-8'\n\n!wget http://download.tensorflow.org/models/object_detection/tf2/20200711/{MODEL3}.tar.gz\n!tar -xf {MODEL3}.tar.gz\n!rm {MODEL3}.tar.gz","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:23.007439Z","iopub.execute_input":"2021-06-12T18:41:23.007923Z","iopub.status.idle":"2021-06-12T18:41:34.571198Z","shell.execute_reply.started":"2021-06-12T18:41:23.007862Z","shell.execute_reply":"2021-06-12T18:41:34.569511Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/working/Tensorflow/workspace/training_demo/pre-trained-models","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:34.573787Z","iopub.execute_input":"2021-06-12T18:41:34.574307Z","iopub.status.idle":"2021-06-12T18:41:35.498479Z","shell.execute_reply.started":"2021-06-12T18:41:34.574254Z","shell.execute_reply":"2021-06-12T18:41:35.497261Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Make a folder models under training_demo","metadata":{}},{"cell_type":"code","source":"!mkdir /kaggle/working/Tensorflow/workspace/training_demo/models","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:35.50281Z","iopub.execute_input":"2021-06-12T18:41:35.5037Z","iopub.status.idle":"2021-06-12T18:41:36.304037Z","shell.execute_reply.started":"2021-06-12T18:41:35.503637Z","shell.execute_reply":"2021-06-12T18:41:36.302601Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"Under models, make the corresponding folders for your pretrained models. Each of your pretrained model subfolder in models fodler will have a modified version of the pipeline.config file which needs to be configured as per the use case.","metadata":{}},{"cell_type":"markdown","source":"presently working for ssd_resent50_fpn","metadata":{}},{"cell_type":"code","source":"!pwd","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:36.307417Z","iopub.execute_input":"2021-06-12T18:41:36.30789Z","iopub.status.idle":"2021-06-12T18:41:37.240466Z","shell.execute_reply.started":"2021-06-12T18:41:36.307838Z","shell.execute_reply":"2021-06-12T18:41:37.239237Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/working/Tensorflow/workspace/training_demo/","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:37.244641Z","iopub.execute_input":"2021-06-12T18:41:37.244949Z","iopub.status.idle":"2021-06-12T18:41:38.031582Z","shell.execute_reply.started":"2021-06-12T18:41:37.244917Z","shell.execute_reply":"2021-06-12T18:41:38.030334Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!mkdir /kaggle/working/Tensorflow/workspace/training_demo/models/ssd_resnet50_v1_fpn_640x640_coco17_tpu-8\n!cp ssd_resnet50_v1_fpn_640x640_coco17_tpu-8/pipeline.config /kaggle/working/Tensorflow/workspace/training_demo/models/ssd_resnet50_v1_fpn_640x640_coco17_tpu-8","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:38.033717Z","iopub.execute_input":"2021-06-12T18:41:38.034231Z","iopub.status.idle":"2021-06-12T18:41:39.608693Z","shell.execute_reply.started":"2021-06-12T18:41:38.034136Z","shell.execute_reply":"2021-06-12T18:41:39.607218Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/working/Tensorflow/workspace/training_demo/models/ssd_resnet50_v1_fpn_640x640_coco17_tpu-8","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:39.612791Z","iopub.execute_input":"2021-06-12T18:41:39.613141Z","iopub.status.idle":"2021-06-12T18:41:40.395317Z","shell.execute_reply.started":"2021-06-12T18:41:39.613106Z","shell.execute_reply":"2021-06-12T18:41:40.39406Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%writefile /kaggle/working/Tensorflow/workspace/training_demo/models/ssd_resnet50_v1_fpn_640x640_coco17_tpu-8/pipeline.config\nmodel {\n  ssd {\n    num_classes: 3 # set this to the number of different label classes\n    image_resizer {\n      fixed_shape_resizer {\n        height: 640\n        width: 640\n      }\n    }\n    feature_extractor {\n      type: \"ssd_resnet50_v1_fpn_keras\"\n      depth_multiplier: 1.0\n      min_depth: 16\n      conv_hyperparams {\n        regularizer {\n          l2_regularizer {\n            weight: 0.00039999998989515007\n          }\n        }\n        initializer {\n          truncated_normal_initializer {\n            mean: 0.0\n            stddev: 0.029999999329447746\n          }\n        }\n        activation: RELU_6\n        batch_norm {\n          decay: 0.996999979019165\n          scale: true\n          epsilon: 0.0010000000474974513\n        }\n      }\n      override_base_feature_extractor_hyperparams: true\n      fpn {\n        min_level: 3\n        max_level: 7\n      }\n    }\n    box_coder {\n      faster_rcnn_box_coder {\n        y_scale: 10.0\n        x_scale: 10.0\n        height_scale: 5.0\n        width_scale: 5.0\n      }\n    }\n    matcher {\n      argmax_matcher {\n        matched_threshold: 0.5\n        unmatched_threshold: 0.5\n        ignore_thresholds: false\n        negatives_lower_than_unmatched: true\n        force_match_for_each_row: true\n        use_matmul_gather: true\n      }\n    }\n    similarity_calculator {\n      iou_similarity {\n      }\n    }\n    box_predictor {\n      weight_shared_convolutional_box_predictor {\n        conv_hyperparams {\n          regularizer {\n            l2_regularizer {\n              weight: 0.00039999998989515007\n            }\n          }\n          initializer {\n            random_normal_initializer {\n              mean: 0.0\n              stddev: 0.009999999776482582\n            }\n          }\n          activation: RELU_6\n          batch_norm {\n            decay: 0.996999979019165\n            scale: true\n            epsilon: 0.0010000000474974513\n          }\n        }\n        depth: 256\n        num_layers_before_predictor: 4\n        kernel_size: 3\n        class_prediction_bias_init: -4.599999904632568\n      }\n    }\n    anchor_generator {\n      multiscale_anchor_generator {\n        min_level: 3\n        max_level: 7\n        anchor_scale: 4.0\n        aspect_ratios: 1.0\n        aspect_ratios: 2.0\n        aspect_ratios: 0.5\n        scales_per_octave: 2\n      }\n    }\n    post_processing {\n      batch_non_max_suppression {\n        score_threshold: 9.99999993922529e-09\n        iou_threshold: 0.6000000238418579\n        max_detections_per_class: 100\n        max_total_detections: 100\n        use_static_shapes: false\n      }\n      score_converter: SIGMOID\n    }\n    normalize_loss_by_num_matches: true\n    loss {\n      localization_loss {\n        weighted_smooth_l1 {\n        }\n      }\n      classification_loss {\n        weighted_sigmoid_focal {\n          gamma: 2.0\n          alpha: 0.25\n        }\n      }\n      classification_weight: 1.0\n      localization_weight: 1.0\n    }\n    encode_background_as_zeros: true\n    normalize_loc_loss_by_codesize: true\n    inplace_batchnorm_update: true\n    freeze_batchnorm: false\n  }\n}\ntrain_config {\n  batch_size: 8   # Increase/Decrease depending on the available memory\n  data_augmentation_options {\n    random_horizontal_flip {\n    }\n  }\n  data_augmentation_options {\n    random_crop_image {\n      min_object_covered: 0.0\n      min_aspect_ratio: 0.75\n      max_aspect_ratio: 3.0\n      min_area: 0.75\n      max_area: 1.0\n      overlap_thresh: 0.0\n    }\n  }\n  sync_replicas: true\n  optimizer {\n    momentum_optimizer {\n      learning_rate {\n        cosine_decay_learning_rate {\n          learning_rate_base: 0.03999999910593033\n          total_steps: 25000\n          warmup_learning_rate: 0.013333000242710114\n          warmup_steps: 2000\n        }\n      }\n      momentum_optimizer_value: 0.8999999761581421\n    }\n    use_moving_average: false\n  }\n  fine_tune_checkpoint: \"/kaggle/working/Tensorflow/workspace/training_demo/pre-trained-models/ssd_resnet50_v1_fpn_640x640_coco17_tpu-8/checkpoint/ckpt-0\"  # Path to checkpoint of pre-trained models\n  num_steps: 2000\n  startup_delay_steps: 0.0\n  replicas_to_aggregate: 8\n  max_number_of_boxes: 100\n  unpad_groundtruth_tensors: false\n  fine_tune_checkpoint_type: \"detection\" # Set this to detection as we want our full model to train for detection\n  use_bfloat16: false # # Set this to false if you are not training on a TPU\n  fine_tune_checkpoint_version: V2\n}\ntrain_input_reader {\n  label_map_path: \"/kaggle/working/Tensorflow/workspace/training_demo/annotations/cells_label_map.pbtxt\"  # Path to label map file\n  tf_record_input_reader {\n    input_path: \"/kaggle/working/Tensorflow/workspace/training_demo/annotations/cells_train.tfrecord\" # Path to training record file\n  }\n}\neval_config {\n  metrics_set: \"coco_detection_metrics\"\n  use_moving_averages: false\n}\neval_input_reader {\n  label_map_path: \"/kaggle/working/Tensorflow/workspace/training_demo/annotations/cells_label_map.pbtxt\"  #Path to label map file\n  shuffle: false\n  num_epochs: 1\n  tf_record_input_reader {\n    input_path: \"/kaggle/working/Tensorflow/workspace/training_demo/annotations/cells_test.tfrecord\"  #Path to testing TFRecord\n  }\n}\n\n","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:40.397541Z","iopub.execute_input":"2021-06-12T18:41:40.39801Z","iopub.status.idle":"2021-06-12T18:41:40.411153Z","shell.execute_reply.started":"2021-06-12T18:41:40.397964Z","shell.execute_reply":"2021-06-12T18:41:40.409641Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/working/Tensorflow/workspace/training_demo/annotations","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:42.705268Z","iopub.execute_input":"2021-06-12T18:41:42.705771Z","iopub.status.idle":"2021-06-12T18:41:43.490256Z","shell.execute_reply.started":"2021-06-12T18:41:42.705738Z","shell.execute_reply":"2021-06-12T18:41:43.48902Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cp /kaggle/working/Tensorflow/models/research/object_detection/model_main_tf2.py /kaggle/working/Tensorflow/workspace/training_demo/","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:46.66291Z","iopub.execute_input":"2021-06-12T18:41:46.663396Z","iopub.status.idle":"2021-06-12T18:41:47.456855Z","shell.execute_reply.started":"2021-06-12T18:41:46.663329Z","shell.execute_reply":"2021-06-12T18:41:47.455521Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%cd /kaggle/working/Tensorflow/workspace/training_demo","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:49.028816Z","iopub.execute_input":"2021-06-12T18:41:49.029223Z","iopub.status.idle":"2021-06-12T18:41:49.038869Z","shell.execute_reply.started":"2021-06-12T18:41:49.029183Z","shell.execute_reply":"2021-06-12T18:41:49.037326Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!python model_main_tf2.py --model_dir=models/ssd_resnet50_v1_fpn_640x640_coco17_tpu-8 --pipeline_config_path=models/ssd_resnet50_v1_fpn_640x640_coco17_tpu-8/pipeline.config","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:41:50.866698Z","iopub.execute_input":"2021-06-12T18:41:50.867098Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/working/Tensorflow/workspace/training_demo/models/ssd_resnet50_v1_fpn_640x640_coco17_tpu-8","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#!ls ./annotations","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.052799Z","iopub.status.idle":"2021-06-12T18:02:09.053403Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#%cd models","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.05452Z","iopub.status.idle":"2021-06-12T18:02:09.055159Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !mkdir {MODEL}\n# !mkdir {MODEL2}","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.056042Z","iopub.status.idle":"2021-06-12T18:02:09.056571Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"# !mkdir models","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.05746Z","iopub.status.idle":"2021-06-12T18:02:09.057991Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !cp /kaggle/working/models/research/object_detection/pre-trained-models/{MODEL}/pipeline.config \\\n#     /kaggle/working/models/research/object_detection/pre-trained-models/{MODEL}/pipeline.config.org","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.058886Z","iopub.status.idle":"2021-06-12T18:02:09.059426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# %cd /kaggle/working/models/research/object_detection\n# !cp /kaggle/working/models/research/object_detection/pre-trained-models/{MODEL}/pipeline.config \\\n#     /kaggle/working/models/research/object_detection/pre-trained-models/{MODEL}/pipeline.config.org","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.060302Z","iopub.status.idle":"2021-06-12T18:02:09.060906Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Prepare your dataset","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# num_classes = 3\n# batch_size = 4\n# ft_checkpoint = \"ssd_resnet50_v1_fpn_640x640_coco17_tpu-8/checkpoint/ckpt-0\"\n# ft_checkpoint_type = \"detection\"\n# tpu = \"false\"\n# labelmap_path = \"labelmap.pbtxt\"\n# training_tfrecord = \"train.record\"\n# testing_tfrecord = \"test.record\"","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.06187Z","iopub.status.idle":"2021-06-12T18:02:09.062394Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"%%writefile pipeline.config\nmodel {\n  ssd {\n    num_classes: {num_classes} # set this to the number of different label classes\n    image_resizer {\n      fixed_shape_resizer {\n        height: 640\n        width: 640\n      }\n    }\n    feature_extractor {\n      type: \"ssd_resnet50_v1_fpn_keras\"\n      depth_multiplier: 1.0\n      min_depth: 16\n      conv_hyperparams {\n        regularizer {\n          l2_regularizer {\n            weight: 0.00039999998989515007\n          }\n        }\n        initializer {\n          truncated_normal_initializer {\n            mean: 0.0\n            stddev: 0.029999999329447746\n          }\n        }\n        activation: RELU_6\n        batch_norm {\n          decay: 0.996999979019165\n          scale: true\n          epsilon: 0.0010000000474974513\n        }\n      }\n      override_base_feature_extractor_hyperparams: true\n      fpn {\n        min_level: 3\n        max_level: 7\n      }\n    }\n    box_coder {\n      faster_rcnn_box_coder {\n        y_scale: 10.0\n        x_scale: 10.0\n        height_scale: 5.0\n        width_scale: 5.0\n      }\n    }\n    matcher {\n      argmax_matcher {\n        matched_threshold: 0.5\n        unmatched_threshold: 0.5\n        ignore_thresholds: false\n        negatives_lower_than_unmatched: true\n        force_match_for_each_row: true\n        use_matmul_gather: true\n      }\n    }\n    similarity_calculator {\n      iou_similarity {\n      }\n    }\n    box_predictor {\n      weight_shared_convolutional_box_predictor {\n        conv_hyperparams {\n          regularizer {\n            l2_regularizer {\n              weight: 0.00039999998989515007\n            }\n          }\n          initializer {\n            random_normal_initializer {\n              mean: 0.0\n              stddev: 0.009999999776482582\n            }\n          }\n          activation: RELU_6\n          batch_norm {\n            decay: 0.996999979019165\n            scale: true\n            epsilon: 0.0010000000474974513\n          }\n        }\n        depth: 256\n        num_layers_before_predictor: 4\n        kernel_size: 3\n        class_prediction_bias_init: -4.599999904632568\n      }\n    }\n    anchor_generator {\n      multiscale_anchor_generator {\n        min_level: 3\n        max_level: 7\n        anchor_scale: 4.0\n        aspect_ratios: 1.0\n        aspect_ratios: 2.0\n        aspect_ratios: 0.5\n        scales_per_octave: 2\n      }\n    }\n    post_processing {\n      batch_non_max_suppression {\n        score_threshold: 9.99999993922529e-09\n        iou_threshold: 0.6000000238418579\n        max_detections_per_class: 100\n        max_total_detections: 100\n        use_static_shapes: false\n      }\n      score_converter: SIGMOID\n    }\n    normalize_loss_by_num_matches: true\n    loss {\n      localization_loss {\n        weighted_smooth_l1 {\n        }\n      }\n      classification_loss {\n        weighted_sigmoid_focal {\n          gamma: 2.0\n          alpha: 0.25\n        }\n      }\n      classification_weight: 1.0\n      localization_weight: 1.0\n    }\n    encode_background_as_zeros: true\n    normalize_loc_loss_by_codesize: true\n    inplace_batchnorm_update: true\n    freeze_batchnorm: false\n  }\n}\ntrain_config {\n  batch_size: {batch_size}   # Increase/Decrease depending on the available memory\n  data_augmentation_options {\n    random_horizontal_flip {\n    }\n  }\n  data_augmentation_options {\n    random_crop_image {\n      min_object_covered: 0.0\n      min_aspect_ratio: 0.75\n      max_aspect_ratio: 3.0\n      min_area: 0.75\n      max_area: 1.0\n      overlap_thresh: 0.0\n    }\n  }\n  sync_replicas: true\n  optimizer {\n    momentum_optimizer {\n      learning_rate {\n        cosine_decay_learning_rate {\n          learning_rate_base: 0.03999999910593033\n          total_steps: 25000\n          warmup_learning_rate: 0.013333000242710114\n          warmup_steps: 2000\n        }\n      }\n      momentum_optimizer_value: 0.8999999761581421\n    }\n    use_moving_average: false\n  }\n  fine_tune_checkpoint: {ft_checkpoint}  # Path to checkpoint of pre-trained models\n  num_steps: 2000\n  startup_delay_steps: 0.0\n  replicas_to_aggregate: 8\n  max_number_of_boxes: 100\n  unpad_groundtruth_tensors: false\n  fine_tune_checkpoint_type: {ft_checkpoint_type} # Set this to detection as we want our full model to train for detection\n  use_bfloat16: {tpu} # # Set this to false if you are not training on a TPU\n  fine_tune_checkpoint_version: V2\n}\ntrain_input_reader {\n  label_map_path: {labelmap_path}  # Path to label map file\n  tf_record_input_reader {\n    input_path: {training_tfrecord} # Path to training record file\n  }\n}\neval_config {\n  metrics_set: \"coco_detection_metrics\"\n  use_moving_averages: false\n}\neval_input_reader {\n  label_map_path: {labelmap_path}  #Path to label map file\n  shuffle: false\n  num_epochs: 1\n  tf_record_input_reader {\n    input_path: {testing_tfrecord}  #Path to testing TFRecord\n  }\n}","metadata":{"execution":{"iopub.status.busy":"2021-06-12T03:54:19.943358Z","iopub.execute_input":"2021-06-12T03:54:19.943669Z","iopub.status.idle":"2021-06-12T03:54:19.95045Z","shell.execute_reply.started":"2021-06-12T03:54:19.943642Z","shell.execute_reply":"2021-06-12T03:54:19.949492Z"}}},{"cell_type":"code","source":"# !python model_main_tf2.py --model_dir=models/my_ssd_resnet50_v1_fpn --pipeline_config_path=models/my_ssd_resnet50_v1_fpn/pipeline.config","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.063255Z","iopub.status.idle":"2021-06-12T18:02:09.063788Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !pip install tf_slim\n# ## Make sure we have `pycocotools` installed\n# !pip -q install pycocotools\n# !pip -q install lvis\n# !pip -q install numba","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.064755Z","iopub.status.idle":"2021-06-12T18:02:09.065295Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import os\n# import pathlib\n\n# # Clone the tensorflow models repository if it doesn't already exist\n# if \"models\" in pathlib.Path.cwd().parts:\n#   while \"models\" in pathlib.Path.cwd().parts:\n#     os.chdir('..')\n# elif not pathlib.Path('models').exists():\n#   !git clone --depth 1 https://github.com/tensorflow/models","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.066192Z","iopub.status.idle":"2021-06-12T18:02:09.066713Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !pwd","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.067628Z","iopub.status.idle":"2021-06-12T18:02:09.068171Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# # Install the Object Detection API\n# %cd models/research/\n# !protoc ./object_detection/protos/*.proto --python_out=.\n# # !cp ./object_detection/packages/tf2/setup.py .\n# # !python -m pip -q install .","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.069038Z","iopub.status.idle":"2021-06-12T18:02:09.069578Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !cp ./object_detection/packages/tf2/setup.py .\n# !python -m pip -q install .","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.070496Z","iopub.status.idle":"2021-06-12T18:02:09.071027Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# os.environ['PYTHONPATH'] += \":/kaggle/working/models\"\n\n# import sys\n# sys.path.append(\"/kaggle/working/models\")","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.071971Z","iopub.status.idle":"2021-06-12T18:02:09.072519Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import numpy as np\n# import os\n# import sys\n# import tarfile\n# import tensorflow as tf\n# import zipfile\n# import cv2\n# import json\n# import pandas as pd\n# import glob\n# import os.path as osp\n# from path import Path\n# import datetime\n# import random\n# import shutil\n# from io import StringIO, BytesIO\n# from PIL import Image\n# from IPython.display import display\n# import re\n\n# %matplotlib inline\n# import matplotlib.pyplot as plt\n# import seaborn as sns\n# from matplotlib import rcParams\n# sns.set(rc={\"font.size\":9,\"axes.titlesize\":15,\"axes.labelsize\":9,\n#             \"axes.titlepad\":11, \"axes.labelpad\":9, \"legend.fontsize\":7,\n#             \"legend.title_fontsize\":7, 'axes.grid' : False})\n\n# from sklearn.model_selection import train_test_split\n\n# ## Import object detection module\n# from object_detection.utils import ops as utils_ops\n# from object_detection.utils import label_map_util\n# from object_detection.utils import visualization_utils as vis_utils\n# from object_detection.protos.string_int_label_map_pb2 import StringIntLabelMap, StringIntLabelMapItem\n# from object_detection.utils import config_util\n# from object_detection.builders import model_builder\n\n# from google.protobuf import text_format\n\n# import tarfile\n# from numba import cuda ","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.073499Z","iopub.status.idle":"2021-06-12T18:02:09.074046Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# # patch tf1 into `utils.ops`\n# utils_ops.tf = tf.compat.v1\n\n# # Patch the location of gfile\n# tf.gfile = tf.io.gfile","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.074973Z","iopub.status.idle":"2021-06-12T18:02:09.075528Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# PATH_TO_LABELS = '/kaggle/working/models/research/object_detection/data/mscoco_label_map.pbtxt'\n# category_index = label_map_util.create_category_index_from_labelmap(PATH_TO_LABELS, use_display_name=True)","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.076415Z","iopub.status.idle":"2021-06-12T18:02:09.076954Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# pretrained_dir = \"/kaggle/working/training_job/model/\"\n\n# if not os.path.exists(pretrained_dir):\n#     os.makedirs(pretrained_dir)\n#     print('Pretrainined Model Directory:', pretrained_dir)","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.077857Z","iopub.status.idle":"2021-06-12T18:02:09.078499Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"MODELS_CONFIG = {\n    'efficientdet-d0': {\n        'model_name': 'efficientdet_d0_coco17_tpu-32',\n        'base_pipeline_file': 'ssd_efficientdet_d0_512x512_coco17_tpu-8.config',\n        'pretrained_checkpoint': 'efficientdet_d0_coco17_tpu-32.tar.gz',\n    },\n    'efficientdet-d1': {\n        'model_name': 'efficientdet_d1_coco17_tpu-32',\n        'base_pipeline_file': 'ssd_efficientdet_d1_640x640_coco17_tpu-8.config',\n        'pretrained_checkpoint': 'efficientdet_d1_coco17_tpu-32.tar.gz',\n    },\n    'efficientdet-d2': {\n        'model_name': 'efficientdet_d2_coco17_tpu-32',\n        'base_pipeline_file': 'ssd_efficientdet_d2_768x768_coco17_tpu-8.config',\n        'pretrained_checkpoint': 'efficientdet_d2_coco17_tpu-32.tar.gz',\n    },\n    'efficientdet-d3': {\n        'model_name': 'efficientdet_d3_coco17_tpu-32',\n        'base_pipeline_file': 'ssd_efficientdet_d3_896x896_coco17_tpu-32.config',\n        'pretrained_checkpoint': 'efficientdet_d3_coco17_tpu-32.tar.gz',\n    },\n    'efficientdet-d4': {\n        'model_name': 'efficientdet_d4_coco17_tpu-32',\n        'base_pipeline_file': 'ssd_efficientdet_d4_896x896_coco17_tpu-32.config',\n        'pretrained_checkpoint': 'efficientdet_d4_coco17_tpu-32.tar.gz',\n    },\n    'efficientdet-d5': {\n        'model_name': 'efficientdet_d5_coco17_tpu-32',\n        'base_pipeline_file': 'ssd_efficientdet_d5_896x896_coco17_tpu-32.config',\n        'pretrained_checkpoint': 'efficientdet_d5_coco17_tpu-32.tar.gz',\n    },\n    'efficientdet-d6': {\n        'model_name': 'efficientdet_d6_coco17_tpu-32',\n        'base_pipeline_file': 'ssd_efficientdet_d6_896x896_coco17_tpu-32.config',\n        'pretrained_checkpoint': 'efficientdet_d6_coco17_tpu-32.tar.gz',\n    },\n    'efficientdet-d7': {\n        'model_name': 'efficientdet_d7_coco17_tpu-32',\n        'base_pipeline_file': 'ssd_efficientdet_d7_896x896_coco17_tpu-32.config',\n        'pretrained_checkpoint': 'efficientdet_d7_coco17_tpu-32.tar.gz',\n    }\n}\n\n## Choosing D1 here for this tutorial\n# chosen_model = 'efficientdet-d1'\n# model_name = MODELS_CONFIG[chosen_model]['model_name']\n# pretrained_checkpoint = MODELS_CONFIG[chosen_model]['pretrained_checkpoint']\n# base_pipeline_file = MODELS_CONFIG[chosen_model]['base_pipeline_file']","metadata":{"execution":{"iopub.status.busy":"2021-06-12T18:02:09.079642Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !ls /kaggle/working/training_job","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# model_url = 'http://download.tensorflow.org/models/object_detection/tf2/20200711/'+pretrained_checkpoint\n# # pretrained_dir = \"/kaggle/working/models/research/object_detection/pretrained\"\n\n# !wget {model_url}\n# !tar -xf {pretrained_checkpoint}\n# !mv {model_name}/ {pretrained_dir}\n# !rm {pretrained_checkpoint}\n\n# model_dir = os.path.join(pretrained_dir, model_name, \"saved_model\")\n# print(\"Pre-trained model directory\", model_dir)\n# model = tf.saved_model.load(str(model_dir))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# model.signatures['serving_default'].output_dtypes","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# model.signatures['serving_default'].output_shapes","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def run_inference_for_single_image(model, image):\n#     '''\n#     Add a wrapper function to call the model, and cleanup the outputs:\n#     '''\n#     image = np.asarray(image)\n#     # The input needs to be a tensor, convert it using `tf.convert_to_tensor`.\n#     input_tensor = tf.convert_to_tensor(image)\n#     # The model expects a batch of images, so add an axis with `tf.newaxis`.\n#     input_tensor = input_tensor[tf.newaxis,...]\n\n#     # Run inference\n#     model_fn = model.signatures['serving_default']\n#     output_dict = model_fn(input_tensor)\n\n#     # All outputs are batches tensors.\n#     # Convert to numpy arrays, and take index [0] to remove the batch dimension.\n#     # We're only interested in the first num_detections.\n#     num_detections = int(output_dict.pop('num_detections'))\n#     output_dict = {key:value[0, :num_detections].numpy() for key,value in output_dict.items()}\n#     output_dict['num_detections'] = num_detections\n\n#     # detection_classes should be ints.\n#     output_dict['detection_classes'] = output_dict['detection_classes'].astype(np.int64)\n\n#     # Handle models with masks:\n#     if 'detection_masks' in output_dict:\n#         # Reframe the the bbox mask to the image size.\n#         detection_masks_reframed = utils_ops.reframe_box_masks_to_image_masks(\n#                   output_dict['detection_masks'], output_dict['detection_boxes'],\n#                    image.shape[0], image.shape[1])      \n#         detection_masks_reframed = tf.cast(detection_masks_reframed > 0.5,\n#                                            tf.uint8)\n#         output_dict['detection_masks_reframed'] = detection_masks_reframed.numpy()\n\n#     return output_dict\n\n# def show_inference(model, image_path):\n#     # the array based representation of the image will be used later in order to prepare the\n#     # result image with boxes and labels on it.\n#     image_np = np.array(Image.open(image_path))\n#     # Actual detection.\n#     output_dict = run_inference_for_single_image(model, image_np)\n#     # Visualization of the results of a detection.\n#     vis_utils.visualize_boxes_and_labels_on_image_array(\n#       image_np,\n#       output_dict['detection_boxes'],\n#       output_dict['detection_classes'],\n#       output_dict['detection_scores'],\n#       category_index,\n#       instance_masks=output_dict.get('detection_masks_reframed', None),\n#       use_normalized_coordinates=True,\n#       line_thickness=8)\n\n#     return image_np\n    \n# def load_image_into_numpy_array(path):\n#     \"\"\"Load an image from file into a numpy array.\n\n#     Puts image into numpy array to feed into tensorflow graph.\n#     Note that by convention we put it into a numpy array with shape\n#     (height, width, channels), where channels=3 for RGB.\n\n#     Args:\n#     path: a file path.\n\n#     Returns:\n#     uint8 numpy array with shape (img_height, img_width, 3)\n#     \"\"\"\n#     img_data = tf.io.gfile.GFile(path, 'rb').read()\n#     image = Image.open(BytesIO(img_data))\n#     (im_width, im_height) = image.size\n#     return np.array(image.getdata()).reshape(\n#       (im_height, im_width, 3)).astype(np.uint8)\n\n# def plot_detections(image_np, boxes, classes, scores, category_index,\n#                     figsize=(12, 16), image_name=None):\n#     \"\"\"Wrapper function to visualize detections.\n\n#     Args:\n#     image_np: uint8 numpy array with shape (img_height, img_width, 3)\n#     boxes: a numpy array of shape [N, 4]\n#     classes: a numpy array of shape [N]. Note that class indices are 1-based,\n#       and match the keys in the label map.\n#     scores: a numpy array of shape [N] or None.  If scores=None, then\n#       this function assumes that the boxes to be plotted are groundtruth\n#       boxes and plot all boxes as black with no classes or scores.\n#     category_index: a dict containing category dictionaries (each holding\n#       category index `id` and category name `name`) keyed by category indices.\n#     figsize: size for the figure.\n#     image_name: a name for the image file.\n#     \"\"\"\n#     image_np_with_annotations = image_np.copy()\n#     viz_utils.visualize_boxes_and_labels_on_image_array(\n#       image_np_with_annotations,\n#       boxes,\n#       classes,\n#       scores,\n#       category_index,\n#       use_normalized_coordinates=True,\n#       min_score_thresh=0.8)\n#     if image_name:\n#         plt.imsave(image_name, image_np_with_annotations)\n#     else:\n#         plt.imshow(image_np_with_annotations)\n        \n        \n# def plot_img(img, size=(18, 18), is_rgb=True, title=\"\", cmap='gray'):\n#     plt.figure(figsize=size)\n#     plt.imshow(img, cmap=cmap)\n#     plt.suptitle(title)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Sample Images","metadata":{}},{"cell_type":"code","source":"# !wget https://cdn.pixabay.com/photo/2015/03/26/09/43/city-690158_960_720.jpg\n# !wget https://cdn.pixabay.com/photo/2017/08/05/23/31/people-2586656_960_720.jpg\n# !wget https://upload.wikimedia.org/wikipedia/commons/6/60/Naxos_Taverna.jpg\n    \n# !wget https://cdn.pixabay.com/photo/2020/02/07/15/33/girl-4827500_960_720.jpg\n# TEST_IMAGE_PATHS = [\n#                     'girl-4827500_960_720.jpg',\n#                     'Naxos_Taverna.jpg',\n#                     'city-690158_960_720.jpg'\n#                    ]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# for image_path in TEST_IMAGE_PATHS:\n#     image = show_inference(model, image_path)\n#     os.remove(image_path)\n#     plot_img(image)\n#     plt.savefig('{}_viz.png'.format(image_path))\n#     plt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# del model\n# device = cuda.get_current_device()\n# device.reset()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# tfr_output_dir = \"/kaggle/working/training_job/tfrecords/\"\n\n# if not os.path.exists(tfr_output_dir):\n#     os.makedirs(tfr_output_dir)\n#     print('TFRecord Directory:', tfr_output_dir)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import matplotlib\n# import matplotlib.pyplot as plt\n\n# import os\n# import random\n# import io\n# import imageio\n# import glob\n# import scipy.misc\n# import numpy as np\n# from six import BytesIO\n# from PIL import Image, ImageDraw, ImageFont\n# from IPython.display import display, Javascript\n# from IPython.display import Image as IPyImage\n\n# import tensorflow as tf\n\n# from object_detection.utils import label_map_util\n# from object_detection.utils import config_util\n# from object_detection.utils import visualization_utils as viz_utils\n# #from object_detection.utils import colab_utils\n# from object_detection.builders import model_builder\n\n# %matplotlib inline","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# #run model builder test\n# !python /kaggle/working/models/research/object_detection/builders/model_builder_tf2_test.py\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def load_image_into_numpy_array(path):\n#   \"\"\"Load an image from file into a numpy array.\n\n#   Puts image into numpy array to feed into tensorflow graph.\n#   Note that by convention we put it into a numpy array with shape\n#   (height, width, channels), where channels=3 for RGB.\n\n#   Args:\n#     path: a file path.\n\n#   Returns:\n#     uint8 numpy array with shape (img_height, img_width, 3)\n#   \"\"\"\n#   img_data = tf.io.gfile.GFile(path, 'rb').read()\n#   image = Image.open(BytesIO(img_data))\n#   (im_width, im_height) = image.size\n#   return np.array(image.getdata()).reshape(\n#       (im_height, im_width, 3)).astype(np.uint8)\n\n# def plot_detections(image_np,\n#                     boxes,\n#                     classes,\n#                     scores,\n#                     category_index,\n#                     figsize=(12, 16),\n#                     image_name=None):\n#   \"\"\"Wrapper function to visualize detections.\n\n#   Args:\n#     image_np: uint8 numpy array with shape (img_height, img_width, 3)\n#     boxes: a numpy array of shape [N, 4]\n#     classes: a numpy array of shape [N]. Note that class indices are 1-based,\n#       and match the keys in the label map.\n#     scores: a numpy array of shape [N] or None.  If scores=None, then\n#       this function assumes that the boxes to be plotted are groundtruth\n#       boxes and plot all boxes as black with no classes or scores.\n#     category_index: a dict containing category dictionaries (each holding\n#       category index `id` and category name `name`) keyed by category indices.\n#     figsize: size for the figure.\n#     image_name: a name for the image file.\n#   \"\"\"\n#   image_np_with_annotations = image_np.copy()\n#   viz_utils.visualize_boxes_and_labels_on_image_array(\n#       image_np_with_annotations,\n#       boxes,\n#       classes,\n#       scores,\n#       category_index,\n#       use_normalized_coordinates=True,\n#       min_score_thresh=0.8)\n#   if image_name:\n#     plt.imsave(image_name, image_np_with_annotations)\n#   else:\n#     plt.imshow(image_np_with_annotations)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# from kaggle_secrets import UserSecretsClient\n# user_secrets = UserSecretsClient()\n# secret_value_2 = user_secrets.get_secret(\"bloodcell_dataset_tf_tfrecords\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# #Downloading data from Roboflow\n# #UPDATE THIS LINK - get our data from Roboflow\n# %cd /kaggle/working\n# !curl -L https://public.roboflow.com/ds/$secret_value_2 > roboflow.zip; unzip roboflow.zip; rm roboflow.zip","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !ls train","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# # NOTE: Update these TFRecord names from \"cells\" and \"cells_label_map\" to your files!\n# test_record_fname = '/kaggle/working/valid/cells.tfrecord'\n# train_record_fname = '/kaggle/working/train/cells.tfrecord'\n# label_map_pbtxt_fname = '/kaggle/working/train/cells_label_map.pbtxt'","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Setting up TF2 Object Detection Model Configurations","metadata":{}},{"cell_type":"code","source":"# ##change chosen model to deploy different models available in the TF2 object detection zoo\n# MODELS_CONFIG = {\n#     'efficientdet-d0': {\n#         'model_name': 'efficientdet_d0_coco17_tpu-32',\n#         'base_pipeline_file': 'ssd_efficientdet_d0_512x512_coco17_tpu-8.config',\n#         'pretrained_checkpoint': 'efficientdet_d0_coco17_tpu-32.tar.gz',\n#         'batch_size': 16\n#     },\n#     'efficientdet-d1': {\n#         'model_name': 'efficientdet_d1_coco17_tpu-32',\n#         'base_pipeline_file': 'ssd_efficientdet_d1_640x640_coco17_tpu-8.config',\n#         'pretrained_checkpoint': 'efficientdet_d1_coco17_tpu-32.tar.gz',\n#         'batch_size': 16\n#     },\n#     'efficientdet-d2': {\n#         'model_name': 'efficientdet_d2_coco17_tpu-32',\n#         'base_pipeline_file': 'ssd_efficientdet_d2_768x768_coco17_tpu-8.config',\n#         'pretrained_checkpoint': 'efficientdet_d2_coco17_tpu-32.tar.gz',\n#         'batch_size': 16\n#     },\n#         'efficientdet-d3': {\n#         'model_name': 'efficientdet_d3_coco17_tpu-32',\n#         'base_pipeline_file': 'ssd_efficientdet_d3_896x896_coco17_tpu-32.config',\n#         'pretrained_checkpoint': 'efficientdet_d3_coco17_tpu-32.tar.gz',\n#         'batch_size': 16\n#     }\n# }\n\n# #in this tutorial we implement the lightweight, smallest state of the art efficientdet model\n# #if you want to scale up tot larger efficientdet models you will likely need more compute!\n# chosen_model = 'efficientdet-d0'\n\n# num_steps = 40000 #The more steps, the longer the training. Increase if your loss function is still decreasing and validation metrics are increasing. \n# num_eval_steps = 500 #Perform evaluation after so many steps\n\n# model_name = MODELS_CONFIG[chosen_model]['model_name']\n# pretrained_checkpoint = MODELS_CONFIG[chosen_model]['pretrained_checkpoint']\n# base_pipeline_file = MODELS_CONFIG[chosen_model]['base_pipeline_file']\n# batch_size = MODELS_CONFIG[chosen_model]['batch_size'] #if you can fit a large batch in memory, it may speed up your training","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# #download pretrained weights\n# %mkdir /kaggle/working/models/research/deploy/\n# %cd /kaggle/working/models/research/deploy/\n\n# import tarfile\n# download_tar = 'http://download.tensorflow.org/models/object_detection/tf2/20200711/' + pretrained_checkpoint\n\n# !wget {download_tar}\n# tar = tarfile.open(pretrained_checkpoint)\n# tar.extractall()\n# tar.close()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# #download base training configuration file\n# %cd /kaggle/working/models/research/deploy\n# download_config = 'https://raw.githubusercontent.com/tensorflow/models/master/research/object_detection/configs/tf2/' + base_pipeline_file\n# !wget {download_config}","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# #prepare\n# pipeline_fname = '/kaggle/working/models/research/deploy/' + base_pipeline_file\n# fine_tune_checkpoint = '/kaggle/working/models/research/deploy/' + model_name + '/checkpoint/ckpt-0'\n\n# def get_num_classes(pbtxt_fname):\n#     from object_detection.utils import label_map_util\n#     label_map = label_map_util.load_labelmap(pbtxt_fname)\n#     categories = label_map_util.convert_label_map_to_categories(\n#         label_map, max_num_classes=90, use_display_name=True)\n#     category_index = label_map_util.create_category_index(categories)\n#     return len(category_index.keys())\n# num_classes = get_num_classes(label_map_pbtxt_fname)\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# #write custom configuration file by slotting our dataset, model checkpoint, and training parameters into the base pipeline file\n\n# import re\n\n# %cd /kaggle/working/models/research/deploy\n# print('writing custom configuration file')\n\n# with open(pipeline_fname) as f:\n#     s = f.read()\n# with open('pipeline_file.config', 'w') as f:\n    \n#     # fine_tune_checkpoint\n#     s = re.sub('fine_tune_checkpoint: \".*?\"',\n#                'fine_tune_checkpoint: \"{}\"'.format(fine_tune_checkpoint), s)\n    \n#     # tfrecord files train and test.\n#     s = re.sub(\n#         '(input_path: \".*?)(PATH_TO_BE_CONFIGURED/train)(.*?\")', 'input_path: \"{}\"'.format(train_record_fname), s)\n#     s = re.sub(\n#         '(input_path: \".*?)(PATH_TO_BE_CONFIGURED/val)(.*?\")', 'input_path: \"{}\"'.format(test_record_fname), s)\n\n#     # label_map_path\n#     s = re.sub(\n#         'label_map_path: \".*?\"', 'label_map_path: \"{}\"'.format(label_map_pbtxt_fname), s)\n\n#     # Set training batch_size.\n#     s = re.sub('batch_size: [0-9]+',\n#                'batch_size: {}'.format(batch_size), s)\n\n#     # Set training steps, num_steps\n#     s = re.sub('num_steps: [0-9]+',\n#                'num_steps: {}'.format(num_steps), s)\n    \n#     # Set number of classes num_classes.\n#     s = re.sub('num_classes: [0-9]+',\n#                'num_classes: {}'.format(num_classes), s)\n    \n#     #fine-tune checkpoint type\n#     s = re.sub(\n#         'fine_tune_checkpoint_type: \"classification\"', 'fine_tune_checkpoint_type: \"{}\"'.format('detection'), s)\n        \n#     f.write(s)\n\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# %cat /kaggle/working/models/research/deploy/pipeline_file.config","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# pipeline_file = '/kaggle/working/models/research/deploy/pipeline_file.config'\n# model_dir = '/kaggle/working/training/'","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !python /kaggle/working/models/research/object_detection/model_main_tf2.py \\\n#     --pipeline_config_path={pipeline_file} \\\n#     --model_dir={model_dir} \\\n#     --alsologtostderr \\\n#     --num_train_steps={num_steps} \\\n#     --sample_1_of_n_eval_examples=1 \\\n#     --num_eval_steps={num_eval_steps}","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# %cd ./models/research","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !protoc object_detection/protos/*.proto --python_out=.\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !cp object_detection/packages/tf2/setup.py . \n# !python -m pip -q install .","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !python object_detection/builders/model_builder_tf2_test.py\n# %cd .. ","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import matplotlib\n# import matplotlib.pyplot as plt\n\n# import os\n# import random\n# import io\n# import imageio\n# import glob\n# import scipy.misc\n# import numpy as np\n# from six import BytesIO\n# from PIL import Image, ImageDraw, ImageFont\n# from IPython.display import display, Javascript\n# from IPython.display import Image as IPyImage\n\n# import tensorflow as tf\n\n# from object_detection.utils import label_map_util\n# from object_detection.utils import config_util\n# from object_detection.utils import visualization_utils as viz_utils\n# #the following one works with google colab only\n# #from object_detection.utils import colab_utils\n# from object_detection.builders import model_builder\n\n# %matplotlib inline","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n# !git clone https://github.com/aalpatya/detect_hands.git\n\n# !cp detect_hands/egohands_dataset_to_csv.py .\n# !python egohands_dataset_to_csv.py","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !cp detect_hands/generate_tfrecord.py .\n# # For the train dataset\n# !python generate_tfrecord.py --csv_input=images/train/train_labels.csv  --output_path=train.record\n# # For the test dataset\n# !python generate_tfrecord.py --csv_input=images/test/test_labels.csv  --output_path=test.record","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !wget http://download.tensorflow.org/models/object_detection/tf2/20200711/ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.tar.gz\n# # Unzip\n# !tar -xzvf ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.tar.gz\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !ls /kaggle/working/models/ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !mv /kaggle/working/models/ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8 /kaggle/working/models/research/object_detection/test_data/","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Utility functions","metadata":{}},{"cell_type":"code","source":"# def load_image_into_numpy_array(path):\n#     \"\"\"Load an image from file into a numpy array.\n\n#     Puts image into numpy array to feed into tensorflow graph.\n#     Note that by convention we put it into a numpy array with shape\n#     (height, width, channels), where channels=3 for RGB.\n\n#     Args:\n#     path: a file path.\n\n#     Returns:\n#     uint8 numpy array with shape (img_height, img_width, 3)\n#     \"\"\"\n    \n#     img_data = tf.io.gfile.GFile(path, 'rb').read()\n#     image = Image.open(BytesIO(img_data))\n#     (im_width, im_height) = image.size\n    \n#     return np.array(image.getdata()).reshape(\n#         (im_height, im_width, 3)).astype(np.uint8)\n\n\n# def plot_detections(image_np,\n#                     boxes,\n#                     classes,\n#                     scores,\n#                     category_index,\n#                     figsize=(12, 16),\n#                     image_name=None):\n#     \"\"\"Wrapper function to visualize detections.\n\n#     Args:\n#     image_np: uint8 numpy array with shape (img_height, img_width, 3)\n#     boxes: a numpy array of shape [N, 4]\n#     classes: a numpy array of shape [N]. Note that class indices are 1-based,\n#           and match the keys in the label map.\n#     scores: a numpy array of shape [N] or None.  If scores=None, then\n#           this function assumes that the boxes to be plotted are groundtruth\n#           boxes and plot all boxes as black with no classes or scores.\n#     category_index: a dict containing category dictionaries (each holding\n#           category index `id` and category name `name`) keyed by category indices.\n#     figsize: size for the figure.\n#     image_name: a name for the image file.\n#     \"\"\"\n    \n#     image_np_with_annotations = image_np.copy()\n    \n#     viz_utils.visualize_boxes_and_labels_on_image_array(\n#         image_np_with_annotations,\n#         boxes,\n#         classes,\n#         scores,\n#         category_index,\n#         use_normalized_coordinates=True,\n#         min_score_thresh=0.8)\n    \n#     if image_name:\n#         plt.imsave(image_name, image_np_with_annotations)\n    \n#     else:\n#         plt.imshow(image_np_with_annotations)\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# # bounding boxes for each of the 5 zombies found in each image. \n# # you can use these instead of drawing the boxes yourself.\n# ref_gt_boxes = [\n#         np.array([[0.27333333, 0.41500586, 0.74333333, 0.57678781]]),\n#         np.array([[0.29833333, 0.45955451, 0.75666667, 0.61078546]]),\n#         np.array([[0.40833333, 0.18288394, 0.945, 0.34818288]]),\n#         np.array([[0.16166667, 0.61899179, 0.8, 0.91910903]]),\n#         np.array([[0.28833333, 0.12543962, 0.835, 0.35052755]]),\n#       ]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Restoring the weights","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Eager Mode training loop","metadata":{}},{"cell_type":"code","source":"# tf.keras.backend.set_learning_phase(True)\n\n# # These parameters can be tuned; since our training set has 5 images\n# # it doesn't make sense to have a much larger batch size, though we could\n# # fit more examples in memory if we wanted to.\n# batch_size = 4\n# learning_rate = 0.01\n# num_batches = 100\n\n# # Select variables in top layers to fine-tune.\n# trainable_variables = detection_model.trainable_variables\n# to_fine_tune = []\n# prefixes_to_train = [\n#   'WeightSharedConvolutionalBoxPredictor/WeightSharedConvolutionalBoxHead',\n#   'WeightSharedConvolutionalBoxPredictor/WeightSharedConvolutionalClassHead']\n# for var in trainable_variables:\n#   if any([var.name.startswith(prefix) for prefix in prefixes_to_train]):\n#     to_fine_tune.append(var)\n\n# # Set up forward + backward pass for a single train step.\n# def get_model_train_step_function(model, optimizer, vars_to_fine_tune):\n#   \"\"\"Get a tf.function for training step.\"\"\"\n\n#   # Use tf.function for a bit of speed.\n#   # Comment out the tf.function decorator if you want the inside of the\n#   # function to run eagerly.\n#   @tf.function\n#   def train_step_fn(image_tensors,\n#                     groundtruth_boxes_list,\n#                     groundtruth_classes_list):\n#     \"\"\"A single training iteration.\n\n#     Args:\n#       image_tensors: A list of [1, height, width, 3] Tensor of type tf.float32.\n#         Note that the height and width can vary across images, as they are\n#         reshaped within this function to be 640x640.\n#       groundtruth_boxes_list: A list of Tensors of shape [N_i, 4] with type\n#         tf.float32 representing groundtruth boxes for each image in the batch.\n#       groundtruth_classes_list: A list of Tensors of shape [N_i, num_classes]\n#         with type tf.float32 representing groundtruth boxes for each image in\n#         the batch.\n\n#     Returns:\n#       A scalar tensor representing the total loss for the input batch.\n#     \"\"\"\n#     shapes = tf.constant(batch_size * [[640, 640, 3]], dtype=tf.int32)\n#     model.provide_groundtruth(\n#         groundtruth_boxes_list=groundtruth_boxes_list,\n#         groundtruth_classes_list=groundtruth_classes_list)\n#     with tf.GradientTape() as tape:\n#       preprocessed_images = tf.concat(\n#           [detection_model.preprocess(image_tensor)[0]\n#            for image_tensor in image_tensors], axis=0)\n#       prediction_dict = model.predict(preprocessed_images, shapes)\n#       losses_dict = model.loss(prediction_dict, shapes)\n#       total_loss = losses_dict['Loss/localization_loss'] + losses_dict['Loss/classification_loss']\n#       gradients = tape.gradient(total_loss, vars_to_fine_tune)\n#       optimizer.apply_gradients(zip(gradients, vars_to_fine_tune))\n#     return total_loss\n\n#   return train_step_fn\n\n# optimizer = tf.keras.optimizers.SGD(learning_rate=learning_rate, momentum=0.9)\n# train_step_fn = get_model_train_step_function(\n#     detection_model, optimizer, to_fine_tune)\n\n# print('Start fine-tuning!', flush=True)\n# for idx in range(num_batches):\n#   # Grab keys for a random subset of examples\n#   all_keys = list(range(len(train_images_np)))\n#   random.shuffle(all_keys)\n#   example_keys = all_keys[:batch_size]\n\n#   # Note that we do not do data augmentation in this demo.  If you want a\n#   # a fun exercise, we recommend experimenting with random horizontal flipping\n#   # and random cropping :)\n#   gt_boxes_list = [gt_box_tensors[key] for key in example_keys]\n#   gt_classes_list = [gt_classes_one_hot_tensors[key] for key in example_keys]\n#   image_tensors = [train_image_tensors[key] for key in example_keys]\n\n#   # Training step (forward pass + backwards pass)\n#   total_loss = train_step_fn(image_tensors, gt_boxes_list, gt_classes_list)\n\n#   if idx % 10 == 0:\n#     print('batch ' + str(idx) + ' of ' + str(num_batches)\n#     + ', loss=' +  str(total_loss.numpy()), flush=True)\n\n# print('Done fine-tuning!')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !cat ./detect_hands/model_data/ssd_mobilenet_v2_fpn_320/label_map.pbtxt","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !cat ./detect_hands/model_data/ssd_mobilenet_v2_fpn_320/pipeline.config","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# %%writefile /kaggle/working/detect_hands/model_data/ssd_mobilenet_v2_fpn_320/pipeline.config\n\n# model {\n#   ssd {\n#     num_classes: 1\n#     image_resizer {\n#       fixed_shape_resizer {\n#         height: 320\n#         width: 320\n#       }\n#     }\n#     feature_extractor {\n#       type: \"ssd_mobilenet_v2_fpn_keras\"\n#       depth_multiplier: 1.0\n#       min_depth: 16\n#       conv_hyperparams {\n#         regularizer {\n#           l2_regularizer {\n#             weight: 4e-05\n#           }\n#         }\n#         initializer {\n#           random_normal_initializer {\n#             mean: 0.0\n#             stddev: 0.01\n#           }\n#         }\n#         activation: RELU_6\n#         batch_norm {\n#           decay: 0.997\n#           scale: true\n#           epsilon: 0.001\n#         }\n#       }\n#       use_depthwise: true\n#       override_base_feature_extractor_hyperparams: true\n#       fpn {\n#         min_level: 3\n#         max_level: 7\n#         additional_layer_depth: 128\n#       }\n#     }\n#     box_coder {\n#       faster_rcnn_box_coder {\n#         y_scale: 10.0\n#         x_scale: 10.0\n#         height_scale: 5.0\n#         width_scale: 5.0\n#       }\n#     }\n#     matcher {\n#       argmax_matcher {\n#         matched_threshold: 0.5\n#         unmatched_threshold: 0.5\n#         ignore_thresholds: false\n#         negatives_lower_than_unmatched: true\n#         force_match_for_each_row: true\n#         use_matmul_gather: true\n#       }\n#     }\n#     similarity_calculator {\n#       iou_similarity {\n#       }\n#     }\n#     box_predictor {\n#       weight_shared_convolutional_box_predictor {\n#         conv_hyperparams {\n#           regularizer {\n#             l2_regularizer {\n#               weight: 4e-05\n#             }\n#           }\n#           initializer {\n#             random_normal_initializer {\n#               mean: 0.0\n#               stddev: 0.01\n#             }\n#           }\n#           activation: RELU_6\n#           batch_norm {\n#             decay: 0.997\n#             scale: true\n#             epsilon: 0.001\n#           }\n#         }\n#         depth: 128\n#         num_layers_before_predictor: 4\n#         kernel_size: 3\n#         class_prediction_bias_init: -4.6\n#         share_prediction_tower: true\n#         use_depthwise: true\n#       }\n#     }\n#     anchor_generator {\n#       multiscale_anchor_generator {\n#         min_level: 3\n#         max_level: 7\n#         anchor_scale: 4.0\n#         aspect_ratios: 1.0\n#         aspect_ratios: 2.0\n#         aspect_ratios: 0.5\n#         scales_per_octave: 2\n#       }\n#     }\n#     post_processing {\n#       batch_non_max_suppression {\n#         score_threshold: 1e-08\n#         iou_threshold: 0.6\n#         max_detections_per_class: 100\n#         max_total_detections: 100\n#         use_static_shapes: false\n#       }\n#       score_converter: SIGMOID\n#     }\n#     normalize_loss_by_num_matches: true\n#     loss {\n#       localization_loss {\n#         weighted_smooth_l1 {\n#         }\n#       }\n#       classification_loss {\n#         weighted_sigmoid_focal {\n#           gamma: 2.0\n#           alpha: 0.25\n#         }\n#       }\n#       classification_weight: 1.0\n#       localization_weight: 1.0\n#     }\n#     encode_background_as_zeros: true\n#     normalize_loc_loss_by_codesize: true\n#     inplace_batchnorm_update: true\n#     freeze_batchnorm: false\n#   }\n# }\n# train_config {\n#    fine_tune_checkpoint_version : V2\n#     fine_tune_checkpoint : \"/kaggle/working/ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8/checkpoint/ckpt-0\"\n#     fine_tune_checkpoint_type : \"detection\" \n#   batch_size: 4\n#   data_augmentation_options {\n#     random_horizontal_flip {\n#     }\n#   }\n#   data_augmentation_options {\n#     random_crop_image {\n#       min_object_covered: 0.0\n#       min_aspect_ratio: 0.75\n#       max_aspect_ratio: 3.0\n#       min_area: 0.75\n#       max_area: 1.0\n#       overlap_thresh: 0.0\n#     }\n#   }\n#   sync_replicas: true\n#   optimizer {\n#     momentum_optimizer {\n#       learning_rate {\n#         cosine_decay_learning_rate {\n#           learning_rate_base: 0.08\n#           total_steps: 50000\n#           warmup_learning_rate: 0.026666\n#           warmup_steps: 1000\n#         }\n#       }\n#       momentum_optimizer_value: 0.9\n#     }\n#     use_moving_average: false\n#   }\n#   num_steps: 50000\n#   startup_delay_steps: 0.0\n#   replicas_to_aggregate: 8\n#   max_number_of_boxes: 100\n#   unpad_groundtruth_tensors: false\n# }\n# train_input_reader {\n#   label_map_path: \"/kaggle/working/detect_hands/model_data/ssd_mobilenet_v2_fpn_320/label_map.pbtxt\"\n#   tf_record_input_reader {\n#     input_path: \"/kaggle/working/train.record\"\n#   }\n# }\n# eval_config {\n#   metrics_set: \"coco_detection_metrics\"\n#   use_moving_averages: false\n# }\n# eval_input_reader {\n#   label_map_path: \"/kaggle/working/detect_hands/model_data/ssd_mobilenet_v2_fpn_320/label_map.pbtxt\"\n#   shuffle: false\n#   num_epochs: 1\n#   tf_record_input_reader {\n#     input_path: \"/kaggle/working/test.record\"\n#   }\n# }","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# %cd ./models/research/object_detection/","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !python model_main_tf2.py \\\n# --pipeline_config_path=/kaggle/working/detect_hands/model_data/ssd_mobilenet_v2_fpn_320/pipeline.config \\\n# --model_dir=/kaggle/working/detect_hands/output_training --alsologtostderr","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### setup","metadata":{}},{"cell_type":"markdown","source":"Activating Tensorboard on Kaggle Notebooks (Latest Docker Images for Kaggle don't support Tensorboard)","metadata":{}},{"cell_type":"code","source":"# import tensorflow as tf\n\n# !rm -rf ./logs/ \n# !mkdir ./logs/","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !wget https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip\n# !unzip ngrok-stable-linux-amd64.zip\n\n# # Run tensorboard as well as Ngrox (for tunneling as non-blocking processes)\n# import os\n# import multiprocessing\n\n\n# pool = multiprocessing.Pool(processes = 10)\n# results_of_processes = [pool.apply_async(os.system, args=(cmd, ), callback = None )\n#                         for cmd in [\n#                         f\"tensorboard --logdir ./logs/ --host 0.0.0.0 --port 6006 &\",\n#                         \"./ngrok http 6006 &\"\n#                         ]]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# ! curl -s http://localhost:4040/api/tunnels | python3 -c \\\n#     \"import sys, json; print(json.load(sys.stdin)['tunnels'][0]['public_url'])\"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# mnist = tf.keras.datasets.mnist\n\n# (x_train, y_train),(x_test, y_test) = mnist.load_data()\n# x_train, x_test = x_train / 255.0, x_test / 255.0\n\n# def create_model():\n#   return tf.keras.models.Sequential([\n#     tf.keras.layers.Flatten(input_shape=(28, 28)),\n#     tf.keras.layers.Dense(512, activation='relu'),\n#     tf.keras.layers.Dropout(0.2),\n#     tf.keras.layers.Dense(10, activation='softmax')\n#   ])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import datetime\n# model = create_model()\n# model.compile(optimizer='adam',\n#               loss='sparse_categorical_crossentropy',\n#               metrics=['accuracy'])\n\n# log_dir = \"logs/fit/\" + datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n# tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)\n\n# model.fit(x=x_train, \n#           y=y_train, \n#           epochs=10, \n#           validation_data=(x_test, y_test), \n#           callbacks=[tensorboard_callback])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### EfficientDet0","metadata":{}},{"cell_type":"markdown","source":"TF2 Object Detection Dependencies","metadata":{}},{"cell_type":"code","source":"# import os\n# import pathlib\n\n# # Clone the tensorflow models repository if it doesn't already exist\n# if \"models\" in pathlib.Path.cwd().parts:\n#   while \"models\" in pathlib.Path.cwd().parts:\n#     os.chdir('..')\n# elif not pathlib.Path('models').exists():\n#   !git clone --depth 1 https://github.com/tensorflow/models","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# # Install the Object Detection API\n# !cd models/research/\n# !protoc object_detection/protos/*.proto --python_out=.\n# !cp object_detection/packages/tf2/setup.py .\n# !python -m pip install .","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### EfficientDet4","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"MASK R-CNN","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Faster RCNN","metadata":{}},{"cell_type":"code","source":"#https://public.roboflow.com/ds/rcXspzRoxm?key=IdrJ79g98i\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !pip install -U -q --pre tensorflow==\"2.2.0\"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import os\n# import pathlib\n\n# # Clone the tensorflow models repository if it doesn't already exist\n# if \"models\" in pathlib.Path.cwd().parts:\n#   while \"models\" in pathlib.Path.cwd().parts:\n#     os.chdir('..')\n# elif not pathlib.Path('models').exists():\n#   !git clone --depth 1 https://github.com/tensorflow/models","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# %cd models\n# %cd research","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# %%bash\n# #cd models/research\n# # Compile protos.\n# protoc object_detection/protos/*.proto --python_out=.\n# cp object_detection/packages/tf2/setup.py .\n# python -m pip install .","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import matplotlib\n# import matplotlib.pyplot as plt\n\n# import os\n# import random\n# import io\n# import imageio\n# import glob\n# import scipy.misc\n# import numpy as np\n# from six import BytesIO\n# from PIL import Image, ImageDraw, ImageFont\n# from IPython.display import display, Javascript\n# from IPython.display import Image as IPyImage\n\n# import tensorflow as tf\n\n# from object_detection.utils import label_map_util\n# from object_detection.utils import config_util\n# from object_detection.utils import visualization_utils as viz_utils\n# #from object_detection.utils import colab_utils\n# from object_detection.builders import model_builder\n\n# %matplotlib inline","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def load_image_into_numpy_array(path):\n#   \"\"\"Load an image from file into a numpy array.\n\n#   Puts image into numpy array to feed into tensorflow graph.\n#   Note that by convention we put it into a numpy array with shape\n#   (height, width, channels), where channels=3 for RGB.\n\n#   Args:\n#     path: a file path.\n\n#   Returns:\n#     uint8 numpy array with shape (img_height, img_width, 3)\n#   \"\"\"\n#   img_data = tf.io.gfile.GFile(path, 'rb').read()\n#   image = Image.open(BytesIO(img_data))\n#   (im_width, im_height) = image.size\n#   return np.array(image.getdata()).reshape(\n#       (im_height, im_width, 3)).astype(np.uint8)\n\n# def plot_detections(image_np,\n#                     boxes,\n#                     classes,\n#                     scores,\n#                     category_index,\n#                     figsize=(12, 16),\n#                     image_name=None):\n#   \"\"\"Wrapper function to visualize detections.\n\n#   Args:\n#     image_np: uint8 numpy array with shape (img_height, img_width, 3)\n#     boxes: a numpy array of shape [N, 4]\n#     classes: a numpy array of shape [N]. Note that class indices are 1-based,\n#       and match the keys in the label map.\n#     scores: a numpy array of shape [N] or None.  If scores=None, then\n#       this function assumes that the boxes to be plotted are groundtruth\n#       boxes and plot all boxes as black with no classes or scores.\n#     category_index: a dict containing category dictionaries (each holding\n#       category index `id` and category name `name`) keyed by category indices.\n#     figsize: size for the figure.\n#     image_name: a name for the image file.\n#   \"\"\"\n#   image_np_with_annotations = image_np.copy()\n#   viz_utils.visualize_boxes_and_labels_on_image_array(\n#       image_np_with_annotations,\n#       boxes,\n#       classes,\n#       scores,\n#       category_index,\n#       use_normalized_coordinates=True,\n#       min_score_thresh=0.8)\n#   if image_name:\n#     plt.imsave(image_name, image_np_with_annotations)\n#   else:\n#     plt.imshow(image_np_with_annotations)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# train_image_dir = '/kaggle/working/models/research/object_detection/test_images/ducky/train/'\n# train_images_np = []\n# for i in range(1, 6):\n#   image_path = os.path.join(train_image_dir, 'robertducky' + str(i) + '.jpg')\n#   train_images_np.append(load_image_into_numpy_array(image_path))\n\n# plt.rcParams['axes.grid'] = False\n# plt.rcParams['xtick.labelsize'] = False\n# plt.rcParams['ytick.labelsize'] = False\n# plt.rcParams['xtick.top'] = False\n# plt.rcParams['xtick.bottom'] = False\n# plt.rcParams['ytick.left'] = False\n# plt.rcParams['ytick.right'] = False\n# plt.rcParams['figure.figsize'] = [14, 7]\n\n# for idx, train_image_np in enumerate(train_images_np):\n#   plt.subplot(2, 3, idx+1)\n#   plt.imshow(train_image_np)\n# plt.show()\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# gt_boxes = [\n#             np.array([[0.436, 0.591, 0.629, 0.712]], dtype=np.float32),\n#             np.array([[0.539, 0.583, 0.73, 0.71]], dtype=np.float32),\n#             np.array([[0.464, 0.414, 0.626, 0.548]], dtype=np.float32),\n#             np.array([[0.313, 0.308, 0.648, 0.526]], dtype=np.float32),\n#             np.array([[0.256, 0.444, 0.484, 0.629]], dtype=np.float32)\n# ]\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# duck_class_id = 1\n# num_classes = 1\n\n# category_index = {duck_class_id: {'id': duck_class_id, 'name': 'rubber_ducky'}}\n\n# # Convert class labels to one-hot; convert everything to tensors.\n# # The `label_id_offset` here shifts all classes by a certain number of indices;\n# # we do this here so that the model receives one-hot labels where non-background\n# # classes start counting at the zeroth index.  This is ordinarily just handled\n# # automatically in our training binaries, but we need to reproduce it here.\n# label_id_offset = 1\n# train_image_tensors = []\n# gt_classes_one_hot_tensors = []\n# gt_box_tensors = []\n# for (train_image_np, gt_box_np) in zip(\n#     train_images_np, gt_boxes):\n#   train_image_tensors.append(tf.expand_dims(tf.convert_to_tensor(\n#       train_image_np, dtype=tf.float32), axis=0))\n#   gt_box_tensors.append(tf.convert_to_tensor(gt_box_np, dtype=tf.float32))\n#   zero_indexed_groundtruth_classes = tf.convert_to_tensor(\n#       np.ones(shape=[gt_box_np.shape[0]], dtype=np.int32) - label_id_offset)\n#   gt_classes_one_hot_tensors.append(tf.one_hot(\n#       zero_indexed_groundtruth_classes, num_classes))\n# print('Done prepping data.')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# dummy_scores = np.array([1.0], dtype=np.float32)  # give boxes a score of 100%\n\n# plt.figure(figsize=(30, 15))\n# for idx in range(5):\n#   plt.subplot(2, 3, idx+1)\n#   plot_detections(\n#       train_images_np[idx],\n#       gt_boxes[idx],\n#       np.ones(shape=[gt_boxes[idx].shape[0]], dtype=np.int32),\n#       dummy_scores, category_index)\n# plt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !ls ssd_resnet50_v1_fpn_640x640_coco17_tpu-8","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !wget http://download.tensorflow.org/models/object_detection/tf2/20200711/ssd_resnet50_v1_fpn_640x640_coco17_tpu-8.tar.gz\n# !tar -xf ssd_resnet50_v1_fpn_640x640_coco17_tpu-8.tar.gz\n# !mv ssd_resnet50_v1_fpn_640x640_coco17_tpu-8/checkpoint /kaggle/working/models/research/object_detection/test_data/","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# tf.keras.backend.clear_session()\n\n# print('Building model and restoring weights for fine-tuning...', flush=True)\n# num_classes = 1\n# pipeline_config = '/kaggle/working/models/research/object_detection/configs/tf2/ssd_resnet50_v1_fpn_640x640_coco17_tpu-8.config'\n# checkpoint_path = '/kaggle/working/models/research/object_detection/test_data/checkpoint/ckpt-0'\n\n# # Load pipeline config and build a detection model.\n# #\n# # Since we are working off of a COCO architecture which predicts 90\n# # class slots by default, we override the `num_classes` field here to be just\n# # one (for our new rubber ducky class).\n# configs = config_util.get_configs_from_pipeline_file(pipeline_config)\n# model_config = configs['model']\n# model_config.ssd.num_classes = num_classes\n# model_config.ssd.freeze_batchnorm = True\n# detection_model = model_builder.build(\n#       model_config=model_config, is_training=True)\n\n# # Set up object-based checkpoint restore --- RetinaNet has two prediction\n# # `heads` --- one for classification, the other for box regression.  We will\n# # restore the box regression head but initialize the classification head\n# # from scratch (we show the omission below by commenting out the line that\n# # we would add if we wanted to restore both heads)\n# fake_box_predictor = tf.compat.v2.train.Checkpoint(\n#     _base_tower_layers_for_heads=detection_model._box_predictor._base_tower_layers_for_heads,\n#     # _prediction_heads=detection_model._box_predictor._prediction_heads,\n#     #    (i.e., the classification head that we *will not* restore)\n#     _box_prediction_head=detection_model._box_predictor._box_prediction_head,\n#     )\n# fake_model = tf.compat.v2.train.Checkpoint(\n#           _feature_extractor=detection_model._feature_extractor,\n#           _box_predictor=fake_box_predictor)\n# ckpt = tf.compat.v2.train.Checkpoint(model=fake_model)\n# ckpt.restore(checkpoint_path).expect_partial()\n\n# # Run model through a dummy image so that variables are created\n# image, shapes = detection_model.preprocess(tf.zeros([1, 640, 640, 3]))\n# prediction_dict = detection_model.predict(image, shapes)\n# _ = detection_model.postprocess(prediction_dict, shapes)\n# print('Weights restored!')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# tf.keras.backend.set_learning_phase(True)\n\n# # These parameters can be tuned; since our training set has 5 images\n# # it doesn't make sense to have a much larger batch size, though we could\n# # fit more examples in memory if we wanted to.\n# batch_size = 4\n# learning_rate = 0.01\n# num_batches = 100\n\n# # Select variables in top layers to fine-tune.\n# trainable_variables = detection_model.trainable_variables\n# to_fine_tune = []\n# prefixes_to_train = [\n#   'WeightSharedConvolutionalBoxPredictor/WeightSharedConvolutionalBoxHead',\n#   'WeightSharedConvolutionalBoxPredictor/WeightSharedConvolutionalClassHead']\n# for var in trainable_variables:\n#   if any([var.name.startswith(prefix) for prefix in prefixes_to_train]):\n#     to_fine_tune.append(var)\n\n# # Set up forward + backward pass for a single train step.\n# def get_model_train_step_function(model, optimizer, vars_to_fine_tune):\n#   \"\"\"Get a tf.function for training step.\"\"\"\n\n#   # Use tf.function for a bit of speed.\n#   # Comment out the tf.function decorator if you want the inside of the\n#   # function to run eagerly.\n#   @tf.function\n#   def train_step_fn(image_tensors,\n#                     groundtruth_boxes_list,\n#                     groundtruth_classes_list):\n#     \"\"\"A single training iteration.\n\n#     Args:\n#       image_tensors: A list of [1, height, width, 3] Tensor of type tf.float32.\n#         Note that the height and width can vary across images, as they are\n#         reshaped within this function to be 640x640.\n#       groundtruth_boxes_list: A list of Tensors of shape [N_i, 4] with type\n#         tf.float32 representing groundtruth boxes for each image in the batch.\n#       groundtruth_classes_list: A list of Tensors of shape [N_i, num_classes]\n#         with type tf.float32 representing groundtruth boxes for each image in\n#         the batch.\n\n#     Returns:\n#       A scalar tensor representing the total loss for the input batch.\n#     \"\"\"\n#     shapes = tf.constant(batch_size * [[640, 640, 3]], dtype=tf.int32)\n#     model.provide_groundtruth(\n#         groundtruth_boxes_list=groundtruth_boxes_list,\n#         groundtruth_classes_list=groundtruth_classes_list)\n#     with tf.GradientTape() as tape:\n#       preprocessed_images = tf.concat(\n#           [detection_model.preprocess(image_tensor)[0]\n#            for image_tensor in image_tensors], axis=0)\n#       prediction_dict = model.predict(preprocessed_images, shapes)\n#       losses_dict = model.loss(prediction_dict, shapes)\n#       total_loss = losses_dict['Loss/localization_loss'] + losses_dict['Loss/classification_loss']\n#       gradients = tape.gradient(total_loss, vars_to_fine_tune)\n#       optimizer.apply_gradients(zip(gradients, vars_to_fine_tune))\n#     return total_loss\n\n#   return train_step_fn\n\n# optimizer = tf.keras.optimizers.SGD(learning_rate=learning_rate, momentum=0.9)\n# train_step_fn = get_model_train_step_function(\n#     detection_model, optimizer, to_fine_tune)\n\n# print('Start fine-tuning!', flush=True)\n# for idx in range(num_batches):\n#   # Grab keys for a random subset of examples\n#   all_keys = list(range(len(train_images_np)))\n#   random.shuffle(all_keys)\n#   example_keys = all_keys[:batch_size]\n\n#   # Note that we do not do data augmentation in this demo.  If you want a\n#   # a fun exercise, we recommend experimenting with random horizontal flipping\n#   # and random cropping :)\n#   gt_boxes_list = [gt_box_tensors[key] for key in example_keys]\n#   gt_classes_list = [gt_classes_one_hot_tensors[key] for key in example_keys]\n#   image_tensors = [train_image_tensors[key] for key in example_keys]\n\n#   # Training step (forward pass + backwards pass)\n#   total_loss = train_step_fn(image_tensors, gt_boxes_list, gt_classes_list)\n\n#   if idx % 10 == 0:\n#     print('batch ' + str(idx) + ' of ' + str(num_batches)\n#     + ', loss=' +  str(total_loss.numpy()), flush=True)\n\n# print('Done fine-tuning!')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# test_image_dir = '/kaggle/working/models/research/object_detection/test_images/ducky/test/'\n# test_images_np = []\n# for i in range(1, 50):\n# \t\timage_path = os.path.join(test_image_dir, 'out' + str(i) + '.jpg')\n# \t\ttest_images_np.append(np.expand_dims(\n# \t\tload_image_into_numpy_array(image_path), axis=0))\n\n# # Again, uncomment this decorator if you want to run inference eagerly\n# @tf.function\n# def detect(input_tensor):\n# \t\tpreprocessed_image, shapes = detection_model.preprocess(input_tensor);prediction_dict = detection_model.predict(preprocessed_image, shapes)\n# \t\treturn detection_model.postprocess(prediction_dict, shapes)\n\n# # Note that the first frame will trigger tracing of the tf.function, which will\n# # take some time, after which inference should be fast.\n\n# label_id_offset = 1\n# for i in range(len(test_images_np)):\n# \t\tinput_tensor = tf.convert_to_tensor(test_images_np[i], dtype=tf.float32)\n# \t\tdetections = detect(input_tensor)\n\n# \t\tplot_detections(test_images_np[i][0], detections['detection_boxes'][0].numpy(), detections['detection_classes'][0].numpy().astype(np.uint32) + label_id_offset,\tdetections['detection_scores'][0].numpy(),category_index, figsize=(15, 20), image_name=\"gif_frame_\" + ('%02d' % i) + \".jpg\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"###### Notebook in making...","metadata":{}},{"cell_type":"markdown","source":"##### Estimated date of completion -> 13th - June - 2021","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}