{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"Welcome to the tutorial. Here we are going to train two different model on the given dataset. YoLo V5 and MobileNet V2 to see which does good, compare the results and proceed.","metadata":{"_uuid":"a0b9b641-960a-46e4-aeb2-ee0fee56aa92","_cell_guid":"414e1588-6ed6-4f0e-b8f9-89a3f8ef80d4","id":"jPZ8Eh8_Difz","trusted":true}},{"cell_type":"markdown","source":"![mobilenet](https://machinethink.net/images/mobilenet-v2/FeatureExtractor@2x.png)","metadata":{"_uuid":"1d9a4e32-e77c-4119-890a-ef671a395ddc","_cell_guid":"001b47b5-e707-428a-b362-d966551a4035","id":"5HPti22yDif2","trusted":true}},{"cell_type":"markdown","source":"**LETS START WITH MOBILE NET FIRST:**\n\nWe create a directory called mobilenet inside which we will keep all our relevant files.","metadata":{"_uuid":"289d32fb-1ae1-442b-b27f-27a4e922656d","_cell_guid":"26f9bba8-1fb4-4fd1-898c-581e9ae77e60","id":"T0nSZ1oJDif3","trusted":true}},{"cell_type":"code","source":"import os\n\nos.mkdir(\"./mobilenet\")","metadata":{"_uuid":"4f20f3c7-98c2-4d4b-acc9-c9b095b38e50","_cell_guid":"255319f2-0976-4f35-88fa-20863721e72a","collapsed":false,"id":"jD2pxhodDif4","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:31:11.893332Z","iopub.execute_input":"2021-12-31T19:31:11.894046Z","iopub.status.idle":"2021-12-31T19:31:11.928035Z","shell.execute_reply.started":"2021-12-31T19:31:11.893922Z","shell.execute_reply":"2021-12-31T19:31:11.927302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we will start by installing tensorflow object detection API.","metadata":{"_uuid":"1235bf62-339c-418b-bacf-98c1915dd08c","_cell_guid":"6a2c0598-ddf5-43ef-b156-f813c7d27b41","id":"nWPZ09FlDif6","trusted":true}},{"cell_type":"markdown","source":"# Step 1: Cloning the repository","metadata":{"_uuid":"0b6ca100-3ef5-434e-b456-bc2bf469a335","_cell_guid":"2203bba8-a194-4aa6-991e-55ff4452d855","id":"FIcE3bc9Dif7","trusted":true}},{"cell_type":"code","source":"!git clone https://github.com/tensorflow/models {'./mobilenet/models'}","metadata":{"_uuid":"5dade78f-9648-488d-9f6b-e2aac433a003","_cell_guid":"80478c25-0ffa-491f-95ec-f74172baf4ee","collapsed":false,"id":"a_HH3FAnDif7","outputId":"ce588160-4005-401f-efad-52b7d808a686","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:31:16.440856Z","iopub.execute_input":"2021-12-31T19:31:16.441107Z","iopub.status.idle":"2021-12-31T19:31:37.723787Z","shell.execute_reply.started":"2021-12-31T19:31:16.441079Z","shell.execute_reply":"2021-12-31T19:31:37.722877Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Step 2: Install protobufs","metadata":{"_uuid":"27eaa95e-c445-4e0d-a564-d2068166bc66","_cell_guid":"b5a62c7d-7c72-4509-a68e-0f042ca5f76f","id":"ffTQj78LDif8","trusted":true}},{"cell_type":"markdown","source":"Using library wget you can download urls. To install protoc either you can donwload the file and set it to the environment variable of os so that we can use the protoc command. We store it in its seperate directory as shown below. Protoc is used to compile .proto files.","metadata":{"_uuid":"876c5bec-10bf-49f2-9aa1-7309191735f5","_cell_guid":"6d62ccb8-9650-4aa1-a771-29d902b14ef5","id":"d0DFLVaEDif9","trusted":true}},{"cell_type":"code","source":"!pip install wget\nimport wget\nurl = \"https://github.com/protocolbuffers/protobuf/releases/download/v3.15.6/protoc-3.15.6-win64.zip\"\nwget.download(url)","metadata":{"_uuid":"c8af30fc-0cc9-4d36-a4f8-5150a187157c","_cell_guid":"ca6383de-f16b-4bd8-873b-85bbd8929453","collapsed":false,"id":"DQE6171zDif-","outputId":"7b856b6f-5266-4091-b1d7-59164a7be894","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:31:48.127183Z","iopub.execute_input":"2021-12-31T19:31:48.127752Z","iopub.status.idle":"2021-12-31T19:31:59.174965Z","shell.execute_reply.started":"2021-12-31T19:31:48.127715Z","shell.execute_reply":"2021-12-31T19:31:59.174135Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.mkdir(\"./mobilenet/protobuf\")\n\n!cp \"./protoc-3.15.6-win64.zip\" \"./mobilenet/protobuf\"\n\n!unzip \"./mobilenet/protobuf/protoc-3.15.6-win64.zip\" -d \"./mobilenet/protobuf\"\n\n# Copy the file to the labeled directory and unzip it.","metadata":{"_uuid":"7f80cf12-08f6-48c6-819c-3a356d5ead60","_cell_guid":"d3f91050-c668-48d7-9f22-44fc56e01209","collapsed":false,"id":"qa5LXg3eDif-","outputId":"abd59977-3456-4748-bfbc-d058e79e9e83","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:31:59.176763Z","iopub.execute_input":"2021-12-31T19:31:59.177112Z","iopub.status.idle":"2021-12-31T19:32:00.542944Z","shell.execute_reply.started":"2021-12-31T19:31:59.177059Z","shell.execute_reply":"2021-12-31T19:32:00.542189Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.environ['PATH'] += os.pathsep + \"./mobilenet/protobuf/bin/protoc.exe\"","metadata":{"_uuid":"52ac6877-5fda-4ebb-b8ae-d265db03aba0","_cell_guid":"12891992-4c43-4b6a-a6f4-f2dbd04b7ff9","collapsed":false,"id":"ZiStPKj2Dif_","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:32:00.544236Z","iopub.execute_input":"2021-12-31T19:32:00.544480Z","iopub.status.idle":"2021-12-31T19:32:00.550179Z","shell.execute_reply.started":"2021-12-31T19:32:00.544450Z","shell.execute_reply":"2021-12-31T19:32:00.549283Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Step 3: Is to compile all the protocs in the object detection directory as shown below and building the object detection API","metadata":{"_uuid":"f65452b6-7b0e-43de-8988-d631570cc7d5","_cell_guid":"27ed0caf-4493-4abb-a8ef-b0143b52bd15","id":"NhfPJoJbDigA","trusted":true}},{"cell_type":"code","source":"!cd ./mobilenet/models/research && protoc object_detection/protos/*.proto --python_out=.\n!cd ./mobilenet/models/research && cp object_detection/packages/tf2/setup.py . && python -m pip install .","metadata":{"_uuid":"c27ca8b1-9739-475d-895f-7511e2cb1ddc","_cell_guid":"6302dc31-a494-403d-8fc8-deb987f1e535","collapsed":false,"id":"UrADheLWDigB","outputId":"71bea078-a5dc-43a2-8239-9cf0d5204d06","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:32:00.552586Z","iopub.execute_input":"2021-12-31T19:32:00.553031Z","iopub.status.idle":"2021-12-31T19:33:38.934751Z","shell.execute_reply.started":"2021-12-31T19:32:00.552993Z","shell.execute_reply":"2021-12-31T19:33:38.933800Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Step 5: Import the object detection API","metadata":{"_uuid":"1900489c-e6ab-4ea8-b3ff-2ffb56235ced","_cell_guid":"969efb35-d3ca-45ef-a729-c45adc0c4258","id":"NnxX4nJBDigB","trusted":true}},{"cell_type":"markdown","source":"Note: If Importing libraries gives error try restarting kernel and import again.","metadata":{"_uuid":"cc707e92-fd1a-40d0-903a-7770703143c0","_cell_guid":"d201523d-d57b-46c1-ad56-3978ede8f0a3","id":"a5lcSl2PDigC","trusted":true}},{"cell_type":"code","source":"import object_detection\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport pandas as pd\nimport tensorflow as tf\nfrom object_detection.utils import dataset_util\nimport ast\nimport cv2\nimport wget\nimport os\nfrom shutil import copyfile \nfrom tqdm.notebook import tqdm\ntqdm.pandas()","metadata":{"_uuid":"68a9498f-8b4b-408d-a15a-03b7c1ead367","_cell_guid":"c049c796-1c5d-4ee4-9e6c-f172cd0cc682","collapsed":false,"id":"s9BTEo4TDigD","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:04.844745Z","iopub.execute_input":"2021-12-31T19:51:04.845122Z","iopub.status.idle":"2021-12-31T19:51:04.851648Z","shell.execute_reply.started":"2021-12-31T19:51:04.845078Z","shell.execute_reply":"2021-12-31T19:51:04.850598Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Step 6: Download the mobile net v2 model from tensorflow.","metadata":{"_uuid":"daad4f80-eb1b-4e5a-b5df-7703ce14770d","_cell_guid":"5084372f-4417-4c9b-a738-97395a262705","id":"OZeu6ydvDigE","trusted":true}},{"cell_type":"markdown","source":"You can get other models [here](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md)\n\nAfter downloading it store it the right directory and unzip it.","metadata":{"_uuid":"30c74d4a-8011-4437-8e84-5adfffb85980","_cell_guid":"44f50a25-3110-49b0-98f1-b91733f367b5","id":"-FK-QFfQDigF","trusted":true}},{"cell_type":"code","source":"model_url = \"http://download.tensorflow.org/models/object_detection/tf2/20200711/ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8.tar.gz\"\nwget.download(model_url)","metadata":{"_uuid":"d93a65a2-da66-4b30-b465-55a61aed5cae","_cell_guid":"a42c9e72-16ee-430d-8787-a5f3c8263718","collapsed":false,"id":"yL2-KRWDDigG","outputId":"348080e7-c8f1-4b9c-c0d4-ab11e675fe37","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:33:38.937923Z","iopub.execute_input":"2021-12-31T19:33:38.938474Z","iopub.status.idle":"2021-12-31T19:33:39.546549Z","shell.execute_reply.started":"2021-12-31T19:33:38.938432Z","shell.execute_reply":"2021-12-31T19:33:39.545659Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.mkdir(\"./mobilenet/my_model\")\nMODEL_NAME = \"ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8.tar.gz\"\n!mv ./ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8.tar.gz ./mobilenet/my_model\n!cd ./mobilenet/my_model && tar -zxvf {MODEL_NAME}","metadata":{"_uuid":"fefc3ac6-8d45-4b2e-83cf-7339597a0597","_cell_guid":"b49ce7df-ed46-47e4-a91f-174d21b515c3","collapsed":false,"id":"JDK1S9mADigG","outputId":"f100f5b2-27c3-4bb2-eda5-251f6f3837e6","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:33:39.547926Z","iopub.execute_input":"2021-12-31T19:33:39.548255Z","iopub.status.idle":"2021-12-31T19:33:41.135909Z","shell.execute_reply.started":"2021-12-31T19:33:39.548218Z","shell.execute_reply":"2021-12-31T19:33:41.135103Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In the dataset, all the images has height and width as 720 and 1280","metadata":{"_uuid":"226064f0-9c02-4bd8-bcf5-0919164e56a9","_cell_guid":"a00feb93-28e8-42ff-abdb-bb643178dffc","id":"-UiG3FzJDigH","trusted":true}},{"cell_type":"code","source":"TRAIN_PATH = '../input/tensorflow-great-barrier-reef/train_images'","metadata":{"_uuid":"096d7c7a-d62a-4dfe-a641-6ab60b67cbfd","_cell_guid":"f750f3d3-83c6-4990-b657-1b3b9d01dc66","collapsed":false,"id":"CPqSZDiLDigH","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:20.160719Z","iopub.execute_input":"2021-12-31T19:51:20.160984Z","iopub.status.idle":"2021-12-31T19:51:20.164884Z","shell.execute_reply.started":"2021-12-31T19:51:20.160954Z","shell.execute_reply":"2021-12-31T19:51:20.164152Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Lets Load the dataset\ndf = pd.read_csv('../input/tensorflow-great-barrier-reef/train.csv')\ndf.head()","metadata":{"_uuid":"756897df-16a6-4bee-9ff1-f2a0e085a991","_cell_guid":"6f6cb971-9050-4824-bc2e-35a7cb72839c","collapsed":false,"id":"IunYWvuRDigH","outputId":"061a510c-a641-4864-c45c-2d81cc8fd427","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:20.449793Z","iopub.execute_input":"2021-12-31T19:51:20.450291Z","iopub.status.idle":"2021-12-31T19:51:20.516779Z","shell.execute_reply.started":"2021-12-31T19:51:20.450253Z","shell.execute_reply":"2021-12-31T19:51:20.516105Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['annotations'].unique()","metadata":{"_uuid":"9cde03f0-d6ce-458c-9de6-7acbb2c77965","_cell_guid":"49fa9612-ea81-46ab-a328-45d6299c3b9e","collapsed":false,"id":"-SkO7Cp8DigI","outputId":"764dadef-5dca-4a51-e9c6-5acf6ea4611a","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:22.703335Z","iopub.execute_input":"2021-12-31T19:51:22.704300Z","iopub.status.idle":"2021-12-31T19:51:22.719899Z","shell.execute_reply.started":"2021-12-31T19:51:22.704249Z","shell.execute_reply":"2021-12-31T19:51:22.719135Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Lets see for how many values we have the annotations\ndf['number_bbox'] = df['annotations'].apply(lambda x: x.count('x'))\ndf.head()","metadata":{"_uuid":"884b2b93-69b0-4d8c-af55-4acc80469129","_cell_guid":"78b8a7da-25bd-47d7-871c-d987d758e1fb","collapsed":false,"id":"5kd7hx0uDigI","outputId":"a899d090-9a63-44b6-acc2-b09b7e324bae","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:24.220954Z","iopub.execute_input":"2021-12-31T19:51:24.221545Z","iopub.status.idle":"2021-12-31T19:51:24.249712Z","shell.execute_reply.started":"2021-12-31T19:51:24.221504Z","shell.execute_reply":"2021-12-31T19:51:24.248930Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['number_bbox'].unique()   # These are the number of bounding boxes","metadata":{"_uuid":"e80b0e05-2361-4814-9417-c78d9189675e","_cell_guid":"4cf0360c-4f5e-4b95-91cc-7da8b7e75b85","collapsed":false,"id":"ZxnIn-1CDigI","outputId":"ae4da7f0-b503-420b-db7a-60991351f047","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:26.131268Z","iopub.execute_input":"2021-12-31T19:51:26.131667Z","iopub.status.idle":"2021-12-31T19:51:26.146712Z","shell.execute_reply.started":"2021-12-31T19:51:26.131630Z","shell.execute_reply":"2021-12-31T19:51:26.139990Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"proportion = lambda x: round(x/len(df) * 100,2)\nnot_available = len(df.loc[df['number_bbox'] == 0])\nproportion(not_available)  \n# almost for 80% of the data we dont have the bounding boxes so we will use only\n# 20% of data to train the model","metadata":{"_uuid":"23e36b1e-0205-4b5c-b52a-53a7790f1aca","_cell_guid":"c800a98c-c1fa-44e1-a1d9-165761a318a9","collapsed":false,"id":"dRAWilyaDigJ","outputId":"ddd93200-0c2f-4ab3-d888-774bd3737694","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:26.528185Z","iopub.execute_input":"2021-12-31T19:51:26.529053Z","iopub.status.idle":"2021-12-31T19:51:26.544996Z","shell.execute_reply.started":"2021-12-31T19:51:26.528969Z","shell.execute_reply":"2021-12-31T19:51:26.544265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Lets seperate those out\ndf_train = df.loc[df['number_bbox'] != 0,:].reset_index(drop=True)\ndf_train.sample(5)","metadata":{"_uuid":"315f2b68-9c35-4e21-b886-d1fcc3670d56","_cell_guid":"abe1728e-a164-42de-b001-c8c8dd425909","collapsed":false,"id":"wDVKpcXFDigJ","outputId":"e1c9f297-3501-44ad-d487-917906aecc16","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:27.963805Z","iopub.execute_input":"2021-12-31T19:51:27.964298Z","iopub.status.idle":"2021-12-31T19:51:27.980952Z","shell.execute_reply.started":"2021-12-31T19:51:27.964257Z","shell.execute_reply":"2021-12-31T19:51:27.980143Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train['annotations'][0]  # Its in form of string we need to use literal_eval","metadata":{"_uuid":"53ce3582-e732-4f9c-9588-32a438d4afe4","_cell_guid":"f2006a4a-84e5-4a41-a5e7-9d16061dae6f","collapsed":false,"id":"Txm6z1Y7DigJ","outputId":"cbb08e92-8dd5-44b8-d82d-eb2315a3a887","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:29.446520Z","iopub.execute_input":"2021-12-31T19:51:29.446974Z","iopub.status.idle":"2021-12-31T19:51:29.452263Z","shell.execute_reply.started":"2021-12-31T19:51:29.446935Z","shell.execute_reply":"2021-12-31T19:51:29.451576Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import ast\n\neval_ = lambda x:ast.literal_eval(x)\ndf_train['annotations'] = df_train['annotations'].progress_apply(eval_)","metadata":{"_uuid":"81de0064-a3c4-40af-a5d4-bd374a1457fe","_cell_guid":"6d5f55e5-23c6-4643-824f-f8d2d22409a8","collapsed":false,"id":"kiF26nswDigJ","outputId":"dd335a24-5f1e-4089-9691-6601d927fed5","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:29.902656Z","iopub.execute_input":"2021-12-31T19:51:29.903131Z","iopub.status.idle":"2021-12-31T19:51:30.326817Z","shell.execute_reply.started":"2021-12-31T19:51:29.903094Z","shell.execute_reply":"2021-12-31T19:51:30.326094Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train['annotations'][0]  # Looks good now","metadata":{"_uuid":"9c70e06c-0999-46ea-a9ba-a14a2997c60f","_cell_guid":"63014156-5915-4e18-b990-6a5af55eea17","collapsed":false,"id":"9tMLuGtbDigJ","outputId":"d97cf4b0-9b25-4489-d590-9ea8e4447a1c","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:31.735098Z","iopub.execute_input":"2021-12-31T19:51:31.735662Z","iopub.status.idle":"2021-12-31T19:51:31.742020Z","shell.execute_reply.started":"2021-12-31T19:51:31.735622Z","shell.execute_reply":"2021-12-31T19:51:31.741124Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n# Now lets first get the coordinates of the bounding boxes\ndef give_bbox_loc(instance):   #instance could be a single dictionary or lists of dictionary\n    return [list(ins.values()) for ins in instance]\n\ndef get_path_image(instance):\n    return os.path.join(TRAIN_PATH,f'video_{instance[\"video_id\"]}',f'{instance[\"video_frame\"]}')\n\ndef load_image(path):\n    return cv2.cvtColor(cv2.imread(path+'.jpg'), cv2.COLOR_BGR2RGB)","metadata":{"_uuid":"e4d0cd2f-b92c-47bc-b71e-b2c589c1dec1","_cell_guid":"b2ed4616-d60c-46b5-b4f2-65269157dd1c","collapsed":false,"id":"pjR303rSDigK","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:34.588064Z","iopub.execute_input":"2021-12-31T19:51:34.588767Z","iopub.status.idle":"2021-12-31T19:51:34.594092Z","shell.execute_reply.started":"2021-12-31T19:51:34.588728Z","shell.execute_reply":"2021-12-31T19:51:34.593269Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train['box_location'] = df_train['annotations'].progress_apply(give_bbox_loc)","metadata":{"_uuid":"844c0eb2-25c3-40ac-809a-c6ce728f8824","_cell_guid":"0e5d0624-704b-45ff-ad0f-cd22d4e70d27","collapsed":false,"id":"7wcM2qWfDigK","outputId":"50200eeb-7e06-4f31-b863-e39af907f5bc","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:34.965192Z","iopub.execute_input":"2021-12-31T19:51:34.965689Z","iopub.status.idle":"2021-12-31T19:51:35.025706Z","shell.execute_reply.started":"2021-12-31T19:51:34.965651Z","shell.execute_reply":"2021-12-31T19:51:35.024873Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.sample(5)","metadata":{"_uuid":"93ae2a89-891b-43b7-9525-3cce01b80a75","_cell_guid":"4a7eecf0-39d3-4261-af77-7cbc0eecb9e0","collapsed":false,"id":"77bUR-nPDigK","outputId":"dba87ad3-d5de-4129-f1b2-343e75eaf8f5","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:36.138144Z","iopub.execute_input":"2021-12-31T19:51:36.138913Z","iopub.status.idle":"2021-12-31T19:51:36.161650Z","shell.execute_reply.started":"2021-12-31T19:51:36.138874Z","shell.execute_reply":"2021-12-31T19:51:36.160752Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train[\"video_id\"].value_counts()","metadata":{"_uuid":"246092cb-ac2d-4511-b98c-c92276c3306c","_cell_guid":"41a6caa2-ebc5-4cf4-a324-10b835b4fc38","collapsed":false,"id":"Qq0VBBceDigK","outputId":"a738915b-2a5d-4918-b5c8-b347132bc16b","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:38.986208Z","iopub.execute_input":"2021-12-31T19:51:38.986820Z","iopub.status.idle":"2021-12-31T19:51:38.996232Z","shell.execute_reply.started":"2021-12-31T19:51:38.986778Z","shell.execute_reply":"2021-12-31T19:51:38.995352Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train['path'] =  df_train.progress_apply(get_path_image, axis=1)","metadata":{"_uuid":"7a78f118-a310-4f3f-98d9-ea937c0130a2","_cell_guid":"0d602c46-a98d-4838-bc19-3a9dac9e89a6","collapsed":false,"id":"_cFcdZHdDigL","outputId":"08358b35-747d-4c21-a9b4-ea4eaacb2344","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:39.478112Z","iopub.execute_input":"2021-12-31T19:51:39.478665Z","iopub.status.idle":"2021-12-31T19:51:39.632717Z","shell.execute_reply.started":"2021-12-31T19:51:39.478628Z","shell.execute_reply":"2021-12-31T19:51:39.628751Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.sample(5)","metadata":{"_uuid":"ac959259-4101-4939-a18b-b713e51900c7","_cell_guid":"7d3a0bbc-abe3-4e26-8505-8e1289551815","collapsed":false,"id":"COH6KdySDigL","outputId":"e2d41314-ea75-4c68-d211-bb02908d51da","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:41.574628Z","iopub.execute_input":"2021-12-31T19:51:41.574898Z","iopub.status.idle":"2021-12-31T19:51:41.595576Z","shell.execute_reply.started":"2021-12-31T19:51:41.574868Z","shell.execute_reply":"2021-12-31T19:51:41.594831Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def normalize_image(bboxes):\n    \n    bboxes = bboxes.astype('float')\n    bboxes[...,[0,2]] = bboxes[...,[0,2]] / 1280   # width is 1280\n    bboxes[...,[1,3]] = bboxes[...,[1,3]] / 720   # height is 720\n    \n    # Now we need to center the image\n    # Simply add half of width and height of image to current x and y it will be the center\n    bboxes[...,[0,1]] = bboxes[...,[0,1]] + bboxes[...,[2,3]] / 2\n    return bboxes\n\n\ndef draw_boxes(image, cordinates, classes = None, legend=False):\n    img = image.copy()\n    for idx in range(len(cordinates)):\n        bbox = cordinates[idx]\n        clas = classes[idx]\n        colors = [np.random.randint(0,255) for _ in range(3)]\n        # Always remeber that we are using yolo format so they are coordinates for center i.e. mid\n        x = round(float(bbox[0]) * img.shape[1],2)\n        y = round(float(bbox[1]) * img.shape[0],2)\n        w = round(float(bbox[2]) * img.shape[1]/2,2)\n        h = round(float(bbox[3]) * img.shape[1]/2,2)\n\n        box = (x-w,y-h,x+w,y+h) # Becasue note that the coordinates comes after the normalize function.\n        r_a = int(box[0])\n        r_b = int(box[1])\n        r_c = int(box[2])\n        r_d = int(box[3])\n        cv2.rectangle(img,(r_a, r_b),(r_c, r_d), thickness = 3, color = colors)\n        cv2.putText(img, clas, org=(r_a, r_d-3),color=(255, 0, 0),\n                    fontFace = cv2.FONT_HERSHEY_SIMPLEX,thickness = 2,fontScale=0.66)\n    return img","metadata":{"_uuid":"c91b0563-0af9-4bc3-aead-6c927879cb44","_cell_guid":"e813a1a1-7efd-4fc5-9899-37828daceece","collapsed":false,"id":"yl2-20dADigL","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:43.830032Z","iopub.execute_input":"2021-12-31T19:51:43.830757Z","iopub.status.idle":"2021-12-31T19:51:43.842460Z","shell.execute_reply.started":"2021-12-31T19:51:43.830717Z","shell.execute_reply":"2021-12-31T19:51:43.841679Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Now lets see some of the datapoints in the image\npoints = df_train.loc[df_train['number_bbox'] >= 8,:].sample(4)\nn_rows = 2\nn_cols = 2\nplt.figure(figsize=(15,15))\nfor row in range(n_rows):\n    for col in range(n_cols):\n        index = row*n_cols + col\n        plt.subplot(n_rows, n_cols, index+1)\n        \n        instance = points.iloc[index]\n       \n        image = load_image(instance['path'])\n        box_cordinates = np.array(instance['box_location'])\n        height = 720\n        width = 1280\n        classes = ['COTS'] * len(box_cordinates)\n        normalized_coordinates = normalize_image(box_cordinates)\n        image = draw_boxes(image,\n                          normalized_coordinates,\n                           classes = classes,\n                           legend=True\n                          )\n        plt.imshow(image)\n        plt.axis('off')\nplt.subplots_adjust(wspace = 0.01, hspace = 0.01)","metadata":{"_uuid":"281ddb8f-5fa5-4e06-912d-b1e82e61647a","_cell_guid":"99c0ef74-d476-4a34-adeb-9acd3959c43e","collapsed":false,"id":"mrz5uExUDigL","outputId":"6f742376-ec8d-47ee-bf48-f12da945254a","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:51:46.160026Z","iopub.execute_input":"2021-12-31T19:51:46.160320Z","iopub.status.idle":"2021-12-31T19:51:47.315544Z","shell.execute_reply.started":"2021-12-31T19:51:46.160289Z","shell.execute_reply":"2021-12-31T19:51:47.314789Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.mkdir(\"./mobilenet/train\")\nos.mkdir(\"./mobilenet/val\")","metadata":{"_uuid":"5e76428d-fe0e-471a-b3c5-7f21c2a857aa","_cell_guid":"8e6aa4a3-cefc-4b7a-b4da-86fe0cac872f","collapsed":false,"id":"JSMIf1r4DigM","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:36:57.113235Z","iopub.execute_input":"2021-12-31T16:36:57.113469Z","iopub.status.idle":"2021-12-31T16:36:57.118229Z","shell.execute_reply.started":"2021-12-31T16:36:57.113439Z","shell.execute_reply":"2021-12-31T16:36:57.117543Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train['fold'] = 'train'\ntrain_instances = int(len(df_train) * 0.8)\nval_instances = len(df_train) - train_instances\nval_index = df_train.sample(val_instances).index\ndf_train.loc[val_index,'fold'] = 'val'","metadata":{"_uuid":"21899e2a-cb3c-4bf4-b73b-7743b1bb317c","_cell_guid":"d307f848-4023-4785-8450-3e3b22b2367c","collapsed":false,"id":"MbglqgOuDigM","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:37:02.017479Z","iopub.execute_input":"2021-12-31T16:37:02.018078Z","iopub.status.idle":"2021-12-31T16:37:02.028305Z","shell.execute_reply.started":"2021-12-31T16:37:02.018035Z","shell.execute_reply":"2021-12-31T16:37:02.027369Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.fold.value_counts()","metadata":{"_uuid":"d8f86a8f-fbdb-4d8c-8fbe-5ece28bb638c","_cell_guid":"aff19ff8-d469-4895-8f37-92a9b0a24f6a","collapsed":false,"id":"dAc0aSiNDigM","outputId":"88006c0e-6a93-401b-86e8-0fa3319d09d7","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:37:03.330705Z","iopub.execute_input":"2021-12-31T16:37:03.330966Z","iopub.status.idle":"2021-12-31T16:37:03.339531Z","shell.execute_reply.started":"2021-12-31T16:37:03.330937Z","shell.execute_reply":"2021-12-31T16:37:03.338417Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for idx in tqdm(range(df_train.shape[0])):\n    instance = df_train.iloc[idx]\n    if instance['fold'] == 'train':\n      copyfile(f'{instance[\"path\"]}.jpg', './mobilenet/train/{}.jpg'.format(instance[\"image_id\"]))\n    else:\n      copyfile(f'{instance[\"path\"]}.jpg', './mobilenet/val/{}.jpg'.format(instance[\"image_id\"]))","metadata":{"_uuid":"67caa968-2b0e-4cda-a3f3-a6904a5fbae1","_cell_guid":"07a4aa10-1204-41f9-9c0c-81791ebcc5ed","collapsed":false,"id":"ZxzixAkzDigM","outputId":"265df677-2e31-4328-ecdd-6bf3d1dfc03e","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:37:05.652279Z","iopub.execute_input":"2021-12-31T16:37:05.652549Z","iopub.status.idle":"2021-12-31T16:38:04.286675Z","shell.execute_reply.started":"2021-12-31T16:37:05.652518Z","shell.execute_reply":"2021-12-31T16:38:04.285949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(len(os.listdir('./mobilenet/val')))\nprint(len(os.listdir('./mobilenet/train')))","metadata":{"_uuid":"6dac205e-9d3e-403e-a48f-66b9b5030cd1","_cell_guid":"8b137790-9fa3-430d-b7ad-c24def667eeb","collapsed":false,"id":"4qWlRC-eDigN","outputId":"a8a3bad0-32d4-4602-dea3-7045ce2b98e1","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:38:29.007970Z","iopub.execute_input":"2021-12-31T16:38:29.008722Z","iopub.status.idle":"2021-12-31T16:38:29.019049Z","shell.execute_reply.started":"2021-12-31T16:38:29.008682Z","shell.execute_reply":"2021-12-31T16:38:29.018188Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Step 7: Creating Annotations","metadata":{"_uuid":"cf480f6a-0075-4e66-b43c-b0f09c837875","_cell_guid":"8f8c114f-f3ac-41e2-a331-21726cf7fe1c","id":"GVja4132DigN","trusted":true}},{"cell_type":"markdown","source":"For every example in your dataset, you should have the following information:\n\n1. An RGB image for the dataset encoded as jpeg or png.\n\n2. A list of bounding boxes for the image. Each bounding box should contain:\nA bounding box coordinates (with origin in top left corner) defined by 4 floating point numbers [ymin, xmin, ymax, xmax]. Note that we store the normalized coordinates (x / width, y / height) in the TFRecord dataset. The class of the object in the bounding box.","metadata":{"_uuid":"9015b2ff-80f1-44cf-833d-3c74f4db6454","_cell_guid":"df6664a2-28b8-45d0-8779-8936473c692f","id":"zaGQto3gDigN","trusted":true}},{"cell_type":"markdown","source":"We will create xml file for every image and store it in the same directory as the image itself.\n\nThe format for our annotations will be as [follows](https://drive.google.com/file/d/1V2rBGit_xQ-jjBMrSg2atIhYCbKqnJ5R/view?usp=sharing)\n You might be wondering why to use extra terms like segmented etc. The point is I used the repository  in my other projects and I didnt want to change lines there so I place something extra here so that things dont change. Important thing is folder,path,size,object and bndbox.","metadata":{"_uuid":"1b6e1fed-5e43-42ac-aa99-1d3190c63d9a","_cell_guid":"418f2c9c-d6fd-4818-8c07-bccb5fef0440","id":"JyGSWBOsDigN","trusted":true}},{"cell_type":"markdown","source":"For creating XML Annotations we will use below xml library. Its very simple to use. Just follow tree like structure.\n\nX = ET.Element('Hello') # This actually creates a root tag 'Hello'\n\nET.SubElement(X,'Bye').text = 'Kaggle' # It creates a subtag within Hello named 'Bye' with value Kaggle.","metadata":{"_uuid":"e46fe0f1-465b-4e24-8886-585a712493f4","_cell_guid":"99a3f231-8557-48c3-a97f-261077f423c7","id":"LdgZv407DigN","trusted":true}},{"cell_type":"code","source":"# Lets now create annotations for our dataset first. I will use .xml format to save the data.\nimport xml.etree.cElementTree as ET\n\nPATH = \"./mobilenet/\"\nfor _, instance in tqdm(df_train.iterrows()):\n  fold = instance['fold']\n  root = ET.Element(\"annotation\")\n  ET.SubElement(root,\"folder\").text = fold\n  ET.SubElement(root,\"filename\").text = instance[\"image_id\"] + '.jpg'\n  ET.SubElement(root,\"path\").text = './mobilenet/{0}/{1}.jpg'.format(fold,instance[\"image_id\"])\n  source = ET.SubElement(root,'source')\n  ET.SubElement(source, 'database').text = 'Unknown'\n  size = ET.SubElement(root,\"size\")\n  ET.SubElement(size, \"width\").text = str(1280)   # We will convert into int later while ggenerating tfrecord.\n  ET.SubElement(size, \"height\").text = str(720)\n  ET.SubElement(size, \"depth\").text = str(3)\n  ET.SubElement(root,'segmented').text = str(0)\n  classname = ET.SubElement(root,\"object\")\n  for element in instance['box_location']:\n    ET.SubElement(classname, \"name\").text = 'COTS'\n    ET.SubElement(classname, \"pose\").text = 'Unspecified'\n    ET.SubElement(classname, \"truncated\").text = '0'\n    ET.SubElement(classname, \"difficult\").text = '0'\n    bndbox = ET.SubElement(classname, \"bndbox\")\n\n    # Dont worry about normalizing it when we will create tfrecord we will\n    # normalize then and there only\n\n    ET.SubElement(bndbox, \"xmin\").text = str(element[0])\n    ET.SubElement(bndbox, \"ymin\").text = str(element[1])\n\n    # If xmax goes out of the boundary we scale it. Very Important\n    xmax = element[0] + element[2]\n    if xmax >= 1280:\n      ET.SubElement(bndbox, \"xmax\").text = str(1280)\n    else:\n      ET.SubElement(bndbox, \"xmax\").text = str(xmax)\n\n    ymax = element[1] + element[3]\n    if ymax >= 720:\n      ET.SubElement(bndbox, \"ymax\").text = str(720) \n    else:   \n      ET.SubElement(bndbox, \"ymax\").text = str(ymax)\n\n  tree = ET.ElementTree(root)\n  tree.write(os.path.join(os.path.join(PATH,fold), instance[\"image_id\"]+'.xml'))","metadata":{"_uuid":"cf4ae27a-a6d5-4f5d-8ec2-3c433e06217d","_cell_guid":"5d81ebe9-6aca-4c86-a6fe-980ea637ac4d","collapsed":false,"id":"ZCNmepxqDigO","outputId":"d2f1a888-b671-4ef8-e8ee-796b64cffc07","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:38:31.687925Z","iopub.execute_input":"2021-12-31T16:38:31.688497Z","iopub.status.idle":"2021-12-31T16:38:33.272690Z","shell.execute_reply.started":"2021-12-31T16:38:31.688457Z","shell.execute_reply":"2021-12-31T16:38:33.271882Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Step 8: Generating TFRecord","metadata":{"_uuid":"f9c7ec24-560e-498a-b36f-ac71998835c9","_cell_guid":"80a2088b-1b65-4d6f-9786-6dbeffc06736","id":"eOSehDgCDigO","trusted":true}},{"cell_type":"markdown","source":"I will be using mthe standard apporach to generate the two tfrecord file one for train and one for val.\n\nSpecial thanks to [Nick](https://github.com/nicknochnack/) for the file\n\nIf you see the repository all we do is to iterate through all the xml file and generating two tfrecord. For getting tfrecord file. We extract certain information like xmin,xmax etc and we encode it it tf_example(in the repository)\n\nTo know how it works refer [here](https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/training.html)","metadata":{"_uuid":"05b48fc0-556d-4386-bc6a-dbee47112368","_cell_guid":"eff71ac6-9dfe-4d17-b1c8-5df602c0887d","id":"LNyN7kL7DigO","trusted":true}},{"cell_type":"code","source":"!git clone https://github.com/Gnopal1132/Generate_TFRecord.git","metadata":{"_uuid":"5f1860ad-29d3-4d67-81c6-edb16f9deab2","_cell_guid":"2da45498-16ec-4ee4-b95e-9b711fa3cf82","collapsed":false,"id":"1B35UbuLDigO","outputId":"9cc1b309-c47a-4ec4-a06c-6ec735bee2fc","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:38:33.730233Z","iopub.execute_input":"2021-12-31T16:38:33.730730Z","iopub.status.idle":"2021-12-31T16:38:35.310618Z","shell.execute_reply.started":"2021-12-31T16:38:33.730690Z","shell.execute_reply":"2021-12-31T16:38:35.309704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.mkdir('./mobilenet/annotations/')","metadata":{"_uuid":"c99ad28d-8df2-40ec-8930-daa7aef17c26","_cell_guid":"6eac3972-3632-4c40-a6e0-fccc0ea7935e","collapsed":false,"id":"gCy3KX6iDigP","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:38:35.312854Z","iopub.execute_input":"2021-12-31T16:38:35.313382Z","iopub.status.idle":"2021-12-31T16:38:35.318546Z","shell.execute_reply.started":"2021-12-31T16:38:35.313340Z","shell.execute_reply":"2021-12-31T16:38:35.317596Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Step 9: Create the Label Map","metadata":{"_uuid":"5c7dee4b-00cc-4aa1-a9fe-18268fee0e58","_cell_guid":"1b2108c8-3938-4fc1-8da9-a3495c6ea17b","id":"T9oG2rysDigP","trusted":true}},{"cell_type":"markdown","source":"Label map simply contains the number of classes and the id","metadata":{"_uuid":"1b25e24e-52b7-4f3b-b288-6dd080621169","_cell_guid":"4c92771d-88c1-4152-8e2e-c201c3790fff","id":"fjhJ73LuDigP","trusted":true}},{"cell_type":"code","source":"labels = [{'name':'COTS', 'id':1}]\nLABEL_MAP = './mobilenet/annotations/label_map.pbtxt'\nwith open(LABEL_MAP, 'w') as f:\n    for label in labels:\n        f.write('item { \\n')\n        f.write('\\tname:\\'{}\\'\\n'.format(label['name']))\n        f.write('\\tid:{}\\n'.format(label['id']))\n        f.write('}\\n')","metadata":{"_uuid":"ae9b4535-e0f5-456f-a76e-afb50abee3be","_cell_guid":"7c2fa963-c00a-4357-997b-c668c3d64e2e","collapsed":false,"id":"2DijorezDigP","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:38:35.551572Z","iopub.execute_input":"2021-12-31T16:38:35.551870Z","iopub.status.idle":"2021-12-31T16:38:35.558090Z","shell.execute_reply.started":"2021-12-31T16:38:35.551837Z","shell.execute_reply":"2021-12-31T16:38:35.556969Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"TF_RECORD_SCRIPT = './Generate_TFRecord/tfrecord.py'\nIMAGE_PATH_TRAIN = \"./mobilenet/train\"\nIMAGE_PATH_VAL = \"./mobilenet/val\"\nANNOTATION_PATH = \"./mobilenet/annotations/\"\nLABELMAP = os.path.join(ANNOTATION_PATH, 'label_map.pbtxt')\n\n!python {TF_RECORD_SCRIPT} -x {IMAGE_PATH_TRAIN} -l {LABELMAP} -o {os.path.join(ANNOTATION_PATH, 'train.record')} \n!python {TF_RECORD_SCRIPT} -x {IMAGE_PATH_VAL} -l {LABELMAP} -o {os.path.join(ANNOTATION_PATH, 'val.record')}","metadata":{"_uuid":"65a329ab-fac5-48b7-bd0c-374c309ce67a","_cell_guid":"82acf57a-f256-419e-86c9-2a29621a865d","collapsed":false,"id":"T182HcvnDigQ","outputId":"2a51ca01-8cb3-4960-c197-312e6e484ac9","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:38:35.908247Z","iopub.execute_input":"2021-12-31T16:38:35.908525Z","iopub.status.idle":"2021-12-31T16:39:08.820253Z","shell.execute_reply.started":"2021-12-31T16:38:35.908494Z","shell.execute_reply":"2021-12-31T16:39:08.819044Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Step 10: Loading the confiuration file,checkpoints and basic settings for training the model","metadata":{"_uuid":"71378ad1-3aed-4f5a-b415-91cc076ef2c6","_cell_guid":"9fff4d5c-951d-4df3-a129-1fa1beb8707d","id":"6iUzBUZADigQ","trusted":true}},{"cell_type":"code","source":"import tensorflow as tf\nfrom object_detection.utils import config_util\nfrom object_detection.protos import pipeline_pb2\nfrom google.protobuf import text_format","metadata":{"_uuid":"435a251a-a3b8-4d14-94a4-f9406b389119","_cell_guid":"13b5928f-ad1d-4538-a486-75e3b50857eb","collapsed":false,"id":"uwDBBxytDigQ","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:40:05.209249Z","iopub.execute_input":"2021-12-31T16:40:05.209811Z","iopub.status.idle":"2021-12-31T16:40:05.214762Z","shell.execute_reply.started":"2021-12-31T16:40:05.209770Z","shell.execute_reply":"2021-12-31T16:40:05.213879Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nMODEL_NAME = 'ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8'\nos.mkdir('./mobilenet/{}'.format(MODEL_NAME))  # This is where we will store our model checkpoint and configuration.\n!cp \"./mobilenet/my_model/ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8/pipeline.config\" \"./mobilenet/ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8\"","metadata":{"_uuid":"aa3db2b8-c545-4978-a0be-271e70d9ef72","_cell_guid":"e222a3be-8e74-47c4-8d47-fd01893a8abd","collapsed":false,"id":"rxkQFEuWDigR","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:40:14.655713Z","iopub.execute_input":"2021-12-31T16:40:14.656021Z","iopub.status.idle":"2021-12-31T16:40:15.321917Z","shell.execute_reply.started":"2021-12-31T16:40:14.655974Z","shell.execute_reply":"2021-12-31T16:40:15.320919Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"PIPELINE_CONFIG = './mobilenet/ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8/pipeline.config' \nconfig = config_util.get_configs_from_pipeline_file(PIPELINE_CONFIG)  # Loading the configuration file","metadata":{"_uuid":"42644205-6245-425a-a7c6-e8ce1cced2af","_cell_guid":"fccd9836-fb8f-4a58-8b49-bb0f3c59b9de","collapsed":false,"id":"fcTyl3n4DigR","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:40:17.928253Z","iopub.execute_input":"2021-12-31T16:40:17.929222Z","iopub.status.idle":"2021-12-31T16:40:17.940265Z","shell.execute_reply.started":"2021-12-31T16:40:17.929173Z","shell.execute_reply":"2021-12-31T16:40:17.939451Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pipeline_config = pipeline_pb2.TrainEvalPipelineConfig()\nwith tf.io.gfile.GFile(PIPELINE_CONFIG, \"r\") as f:                                                                                                                                                                                                                     \n    proto_str = f.read()                                                                                                                                                                                                                                          \n    text_format.Merge(proto_str, pipeline_config)","metadata":{"_uuid":"0956e09d-c152-4beb-b8e7-33490f386fac","_cell_guid":"c7bf3881-9897-4bc2-95f6-39f87fd03f5e","collapsed":false,"id":"Rmo-snQ6DigR","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:40:19.563845Z","iopub.execute_input":"2021-12-31T16:40:19.564643Z","iopub.status.idle":"2021-12-31T16:40:19.575294Z","shell.execute_reply.started":"2021-12-31T16:40:19.564604Z","shell.execute_reply":"2021-12-31T16:40:19.574394Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"BATCH = 8\nEPOCHS = 2000","metadata":{"_uuid":"f0acfb7f-6ff6-418b-8588-51d07974bf64","_cell_guid":"0807ec27-4f4e-4e7c-8aec-db423e733fb3","collapsed":false,"id":"fPR7QTOqDigR","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:40:21.052603Z","iopub.execute_input":"2021-12-31T16:40:21.052874Z","iopub.status.idle":"2021-12-31T16:40:21.057459Z","shell.execute_reply.started":"2021-12-31T16:40:21.052843Z","shell.execute_reply":"2021-12-31T16:40:21.056302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Loading basic information to the pipeline\nPATH = './mobilenet'\npipeline_config.model.ssd.num_classes = 1\npipeline_config.train_config.batch_size = BATCH\npipeline_config.train_config.fine_tune_checkpoint = os.path.join(os.path.join(PATH, 'my_model'), MODEL_NAME, 'checkpoint', 'ckpt-0')\npipeline_config.train_config.fine_tune_checkpoint_type = \"detection\"\npipeline_config.train_input_reader.label_map_path= os.path.join(PATH,'annotations','label_map.pbtxt')\npipeline_config.train_input_reader.tf_record_input_reader.input_path[:] = [os.path.join(os.path.join(PATH,'annotations'), 'train.record')]\npipeline_config.eval_input_reader[0].label_map_path = os.path.join(PATH,'annotations','label_map.pbtxt')\npipeline_config.eval_input_reader[0].tf_record_input_reader.input_path[:] = [os.path.join(os.path.join(PATH,'annotations'), 'val.record')]","metadata":{"_uuid":"ff5c3db7-3f15-4873-bca7-2fcc47d86b68","_cell_guid":"a63de702-616f-4d34-b070-f8613eb49e6f","collapsed":false,"id":"YXjTEmRLDigR","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:40:22.387373Z","iopub.execute_input":"2021-12-31T16:40:22.388253Z","iopub.status.idle":"2021-12-31T16:40:22.396536Z","shell.execute_reply.started":"2021-12-31T16:40:22.388201Z","shell.execute_reply":"2021-12-31T16:40:22.395822Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"config_text = text_format.MessageToString(pipeline_config)                                                                                                                                                                                                        \nwith tf.io.gfile.GFile(PIPELINE_CONFIG, \"wb\") as f:                                                                                                                                                                                                                     \n    f.write(config_text)","metadata":{"_uuid":"d55c324e-532c-4dd9-9b55-f743d82f1e0a","_cell_guid":"c6888dca-6825-480e-8607-7a58f409f564","collapsed":false,"id":"bomSE25FDigS","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:40:24.670828Z","iopub.execute_input":"2021-12-31T16:40:24.671400Z","iopub.status.idle":"2021-12-31T16:40:24.677633Z","shell.execute_reply.started":"2021-12-31T16:40:24.671354Z","shell.execute_reply":"2021-12-31T16:40:24.676825Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"TRAINING_SCRIPT = os.path.join(\"./mobilenet/models\", 'research', 'object_detection', 'model_main_tf2.py')","metadata":{"_uuid":"afc57d42-1a0e-4f47-9828-464c39904c23","_cell_guid":"d35d31fe-993f-4f27-bb6c-7fa7b32b8e1f","collapsed":false,"id":"hcIT8u3CDigS","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:40:26.080899Z","iopub.execute_input":"2021-12-31T16:40:26.081794Z","iopub.status.idle":"2021-12-31T16:40:26.086728Z","shell.execute_reply.started":"2021-12-31T16:40:26.081753Z","shell.execute_reply":"2021-12-31T16:40:26.085884Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"CHECKPOINT_PATH = './mobilenet/ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8'\ncommand = \"python {} --model_dir={} --pipeline_config_path={} --num_train_steps={}\".format(TRAINING_SCRIPT, CHECKPOINT_PATH, PIPELINE_CONFIG, EPOCHS)","metadata":{"_uuid":"bcbe532a-a7b4-4727-84d3-7d917ac4633d","_cell_guid":"1052700e-7f01-44b6-9383-7cbf5e4490f2","collapsed":false,"id":"USso5F9jDigS","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:40:27.602574Z","iopub.execute_input":"2021-12-31T16:40:27.603136Z","iopub.status.idle":"2021-12-31T16:40:27.607321Z","shell.execute_reply.started":"2021-12-31T16:40:27.603094Z","shell.execute_reply":"2021-12-31T16:40:27.606362Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"command","metadata":{"_uuid":"b1b9d0a0-bdc6-4e26-8989-156c8192ff23","_cell_guid":"19e9529b-9d05-4ece-be78-d0ac04478415","collapsed":false,"id":"eWallLbRDigS","outputId":"9e93b887-bc82-4a11-a445-3726b5fb1db6","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T16:40:27.992678Z","iopub.execute_input":"2021-12-31T16:40:27.992964Z","iopub.status.idle":"2021-12-31T16:40:28.001227Z","shell.execute_reply.started":"2021-12-31T16:40:27.992931Z","shell.execute_reply":"2021-12-31T16:40:28.000217Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To Execute the model run the below line. I am not going to train the model. As I already did it on colab I am going to print the highlight of training and tensorboard to see the output of how the model behaved. I uploaded the final checkpoints and event files.","metadata":{}},{"cell_type":"code","source":"!python ./mobilenet/models/research/object_detection/model_main_tf2.py --model_dir=./mobilenet/ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8 --pipeline_config_path=./mobilenet/ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8/pipeline.config --num_train_steps=2000","metadata":{"_uuid":"ac752d62-eddb-4c80-96f7-dd59efccb34c","_cell_guid":"e632c53c-db7f-44e7-8ea4-1d422c16df3a","collapsed":false,"id":"xlaMXr1PDigS","outputId":"5af26645-00f8-43e5-f0e3-bc5ca7725392","jupyter":{"outputs_hidden":false},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\nINFO:tensorflow:Step 100 per-step time 2.361s\n\nI1230 19:11:27.819197 140299792009088 model_lib_v2.py:707] Step 100 per-step time 2.361s\nINFO:tensorflow:{'Loss/classification_loss': 0.53248906,\n 'Loss/localization_loss': 0.4413896,\n 'Loss/regularization_loss': 0.1521543,\n 'Loss/total_loss': 1.126033,\n 'learning_rate': 0.0319994}\nI1230 19:11:27.819756 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.53248906,\n 'Loss/localization_loss': 0.4413896,\n 'Loss/regularization_loss': 0.1521543,\n 'Loss/total_loss': 1.126033,\n 'learning_rate': 0.0319994}\n \n \nINFO:tensorflow:Step 200 per-step time 1.842s\nI1230 19:14:31.999863 140299792009088 model_lib_v2.py:707] Step 200 per-step time 1.842s\nINFO:tensorflow:{'Loss/classification_loss': 0.57567745,\n 'Loss/localization_loss': 0.420439,\n 'Loss/regularization_loss': 0.15256938,\n 'Loss/total_loss': 1.1486858,\n 'learning_rate': 0.0373328}\nI1230 19:14:32.000326 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.57567745,\n 'Loss/localization_loss': 0.420439,\n 'Loss/regularization_loss': 0.15256938,\n 'Loss/total_loss': 1.1486858,\n 'learning_rate': 0.0373328}\n \n \nINFO:tensorflow:Step 300 per-step time 1.867s\nI1230 19:17:38.737374 140299792009088 model_lib_v2.py:707] Step 300 per-step time 1.867s\nINFO:tensorflow:{'Loss/classification_loss': 0.82316196,\n 'Loss/localization_loss': 0.38124648,\n 'Loss/regularization_loss': 0.15306622,\n 'Loss/total_loss': 1.3574746,\n 'learning_rate': 0.0426662}\nI1230 19:17:38.737965 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.82316196,\n 'Loss/localization_loss': 0.38124648,\n 'Loss/regularization_loss': 0.15306622,\n 'Loss/total_loss': 1.3574746,\n 'learning_rate': 0.0426662}\n \n \nINFO:tensorflow:Step 400 per-step time 1.853s\nI1230 19:20:44.022718 140299792009088 model_lib_v2.py:707] Step 400 per-step time 1.853s\nINFO:tensorflow:{'Loss/classification_loss': 0.5168445,\n 'Loss/localization_loss': 0.30190247,\n 'Loss/regularization_loss': 0.15343656,\n 'Loss/total_loss': 0.9721835,\n 'learning_rate': 0.047999598}\nI1230 19:20:44.023141 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.5168445,\n 'Loss/localization_loss': 0.30190247,\n 'Loss/regularization_loss': 0.15343656,\n 'Loss/total_loss': 0.9721835,\n 'learning_rate': 0.047999598}\n \n \nINFO:tensorflow:Step 500 per-step time 1.865s\nI1230 19:23:50.505155 140299792009088 model_lib_v2.py:707] Step 500 per-step time 1.865s\nINFO:tensorflow:{'Loss/classification_loss': 0.50315356,\n 'Loss/localization_loss': 0.4802518,\n 'Loss/regularization_loss': 0.15407632,\n 'Loss/total_loss': 1.1374817,\n 'learning_rate': 0.053333}\nI1230 19:23:50.505642 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.50315356,\n 'Loss/localization_loss': 0.4802518,\n 'Loss/regularization_loss': 0.15407632,\n 'Loss/total_loss': 1.1374817,\n 'learning_rate': 0.053333}\n \n \nINFO:tensorflow:Step 600 per-step time 1.853s\nI1230 19:26:55.792762 140299792009088 model_lib_v2.py:707] Step 600 per-step time 1.853s\nINFO:tensorflow:{'Loss/classification_loss': 0.2856278,\n 'Loss/localization_loss': 0.15508056,\n 'Loss/regularization_loss': 0.15469529,\n 'Loss/total_loss': 0.5954037,\n 'learning_rate': 0.0586664}\nI1230 19:26:55.793148 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.2856278,\n 'Loss/localization_loss': 0.15508056,\n 'Loss/regularization_loss': 0.15469529,\n 'Loss/total_loss': 0.5954037,\n 'learning_rate': 0.0586664}\n \n \nINFO:tensorflow:Step 700 per-step time 1.856s\nI1230 19:30:01.431559 140299792009088 model_lib_v2.py:707] Step 700 per-step time 1.856s\nINFO:tensorflow:{'Loss/classification_loss': 0.44273925,\n 'Loss/localization_loss': 0.26241776,\n 'Loss/regularization_loss': 0.15521659,\n 'Loss/total_loss': 0.8603736,\n 'learning_rate': 0.0639998}\nI1230 19:30:01.432031 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.44273925,\n 'Loss/localization_loss': 0.26241776,\n 'Loss/regularization_loss': 0.15521659,\n 'Loss/total_loss': 0.8603736,\n 'learning_rate': 0.0639998}\n \n \nINFO:tensorflow:Step 800 per-step time 1.853s\nI1230 19:33:06.761075 140299792009088 model_lib_v2.py:707] Step 800 per-step time 1.853s\nINFO:tensorflow:{'Loss/classification_loss': 0.7442434,\n 'Loss/localization_loss': 0.26918525,\n 'Loss/regularization_loss': 0.15582325,\n 'Loss/total_loss': 1.1692519,\n 'learning_rate': 0.069333196}\nI1230 19:33:06.761445 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.7442434,\n 'Loss/localization_loss': 0.26918525,\n 'Loss/regularization_loss': 0.15582325,\n 'Loss/total_loss': 1.1692519,\n 'learning_rate': 0.069333196}\n \n \nINFO:tensorflow:Step 900 per-step time 1.848s\nI1230 19:36:11.598434 140299792009088 model_lib_v2.py:707] Step 900 per-step time 1.848s\nINFO:tensorflow:{'Loss/classification_loss': 0.47391683,\n 'Loss/localization_loss': 0.23772612,\n 'Loss/regularization_loss': 0.1564635,\n 'Loss/total_loss': 0.8681065,\n 'learning_rate': 0.074666604}\nI1230 19:36:11.598908 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.47391683,\n 'Loss/localization_loss': 0.23772612,\n 'Loss/regularization_loss': 0.1564635,\n 'Loss/total_loss': 0.8681065,\n 'learning_rate': 0.074666604}\n \n \nINFO:tensorflow:Step 1000 per-step time 1.866s\nI1230 19:39:18.184226 140299792009088 model_lib_v2.py:707] Step 1000 per-step time 1.866s\nINFO:tensorflow:{'Loss/classification_loss': 0.56101376,\n 'Loss/localization_loss': 0.32822332,\n 'Loss/regularization_loss': 0.1571509,\n 'Loss/total_loss': 1.0463879,\n 'learning_rate': 0.08}\nI1230 19:39:18.184702 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.56101376,\n 'Loss/localization_loss': 0.32822332,\n 'Loss/regularization_loss': 0.1571509,\n 'Loss/total_loss': 1.0463879,\n 'learning_rate': 0.08}\n \n \nINFO:tensorflow:Step 1100 per-step time 1.857s\nI1230 19:42:23.911181 140299792009088 model_lib_v2.py:707] Step 1100 per-step time 1.857s\nINFO:tensorflow:{'Loss/classification_loss': 0.34899262,\n 'Loss/localization_loss': 0.29847386,\n 'Loss/regularization_loss': 0.15807542,\n 'Loss/total_loss': 0.8055419,\n 'learning_rate': 0.07999918}\nI1230 19:42:23.911592 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.34899262,\n 'Loss/localization_loss': 0.29847386,\n 'Loss/regularization_loss': 0.15807542,\n 'Loss/total_loss': 0.8055419,\n 'learning_rate': 0.07999918}\n \n \nINFO:tensorflow:Step 1200 per-step time 1.846s\nI1230 19:45:28.488472 140299792009088 model_lib_v2.py:707] Step 1200 per-step time 1.846s\nINFO:tensorflow:{'Loss/classification_loss': 0.4211598,\n 'Loss/localization_loss': 0.2639899,\n 'Loss/regularization_loss': 0.15880023,\n 'Loss/total_loss': 0.8439499,\n 'learning_rate': 0.079996705}\nI1230 19:45:28.488929 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.4211598,\n 'Loss/localization_loss': 0.2639899,\n 'Loss/regularization_loss': 0.15880023,\n 'Loss/total_loss': 0.8439499,\n 'learning_rate': 0.079996705}\n \n \nINFO:tensorflow:Step 1300 per-step time 1.846s\nI1230 19:48:33.054533 140299792009088 model_lib_v2.py:707] Step 1300 per-step time 1.846s\nINFO:tensorflow:{'Loss/classification_loss': 0.27099043,\n 'Loss/localization_loss': 0.2084263,\n 'Loss/regularization_loss': 0.15907411,\n 'Loss/total_loss': 0.63849086,\n 'learning_rate': 0.0799926}\nI1230 19:48:33.054928 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.27099043,\n 'Loss/localization_loss': 0.2084263,\n 'Loss/regularization_loss': 0.15907411,\n 'Loss/total_loss': 0.63849086,\n 'learning_rate': 0.0799926}\n \n \nINFO:tensorflow:Step 1400 per-step time 1.841s\nI1230 19:51:37.191095 140299792009088 model_lib_v2.py:707] Step 1400 per-step time 1.841s\nINFO:tensorflow:{'Loss/classification_loss': 0.30937624,\n 'Loss/localization_loss': 0.16050465,\n 'Loss/regularization_loss': 0.15938759,\n 'Loss/total_loss': 0.62926847,\n 'learning_rate': 0.07998685}\nI1230 19:51:37.191522 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.30937624,\n 'Loss/localization_loss': 0.16050465,\n 'Loss/regularization_loss': 0.15938759,\n 'Loss/total_loss': 0.62926847,\n 'learning_rate': 0.07998685}\n \n \nINFO:tensorflow:Step 1500 per-step time 1.867s\nI1230 19:54:43.897237 140299792009088 model_lib_v2.py:707] Step 1500 per-step time 1.867s\nINFO:tensorflow:{'Loss/classification_loss': 0.3307212,\n 'Loss/localization_loss': 0.24422823,\n 'Loss/regularization_loss': 0.16004217,\n 'Loss/total_loss': 0.7349916,\n 'learning_rate': 0.07997945}\nI1230 19:54:43.897655 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.3307212,\n 'Loss/localization_loss': 0.24422823,\n 'Loss/regularization_loss': 0.16004217,\n 'Loss/total_loss': 0.7349916,\n 'learning_rate': 0.07997945}\n \n \nINFO:tensorflow:Step 1600 per-step time 1.844s\nI1230 19:57:48.281800 140299792009088 model_lib_v2.py:707] Step 1600 per-step time 1.844s\nINFO:tensorflow:{'Loss/classification_loss': 0.32862264,\n 'Loss/localization_loss': 0.19054511,\n 'Loss/regularization_loss': 0.16015424,\n 'Loss/total_loss': 0.679322,\n 'learning_rate': 0.079970405}\nI1230 19:57:48.282171 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.32862264,\n 'Loss/localization_loss': 0.19054511,\n 'Loss/regularization_loss': 0.16015424,\n 'Loss/total_loss': 0.679322,\n 'learning_rate': 0.079970405}\n \n \nINFO:tensorflow:Step 1700 per-step time 1.866s\nI1230 20:00:54.930995 140299792009088 model_lib_v2.py:707] Step 1700 per-step time 1.866s\nINFO:tensorflow:{'Loss/classification_loss': 0.30334967,\n 'Loss/localization_loss': 0.24827217,\n 'Loss/regularization_loss': 0.16041782,\n 'Loss/total_loss': 0.7120397,\n 'learning_rate': 0.07995972}\nI1230 20:00:54.931398 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.30334967,\n 'Loss/localization_loss': 0.24827217,\n 'Loss/regularization_loss': 0.16041782,\n 'Loss/total_loss': 0.7120397,\n 'learning_rate': 0.07995972}\n \n \nINFO:tensorflow:Step 1800 per-step time 1.878s\nI1230 20:04:02.684715 140299792009088 model_lib_v2.py:707] Step 1800 per-step time 1.878s\nINFO:tensorflow:{'Loss/classification_loss': 0.29834068,\n 'Loss/localization_loss': 0.1236572,\n 'Loss/regularization_loss': 0.16077736,\n 'Loss/total_loss': 0.58277524,\n 'learning_rate': 0.0799474}\nI1230 20:04:02.685132 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.29834068,\n 'Loss/localization_loss': 0.1236572,\n 'Loss/regularization_loss': 0.16077736,\n 'Loss/total_loss': 0.58277524,\n 'learning_rate': 0.0799474}\n \n \nINFO:tensorflow:Step 1900 per-step time 1.853s\nI1230 20:07:07.961340 140299792009088 model_lib_v2.py:707] Step 1900 per-step time 1.853s\nINFO:tensorflow:{'Loss/classification_loss': 0.3075142,\n 'Loss/localization_loss': 0.17589329,\n 'Loss/regularization_loss': 0.16102299,\n 'Loss/total_loss': 0.6444305,\n 'learning_rate': 0.07993342}\nI1230 20:07:07.961811 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.3075142,\n 'Loss/localization_loss': 0.17589329,\n 'Loss/regularization_loss': 0.16102299,\n 'Loss/total_loss': 0.6444305,\n 'learning_rate': 0.07993342}\n \n \nINFO:tensorflow:Step 2000 per-step time 1.860s\nI1230 20:10:13.916106 140299792009088 model_lib_v2.py:707] Step 2000 per-step time 1.860s\nINFO:tensorflow:{'Loss/classification_loss': 0.18447536,\n 'Loss/localization_loss': 0.14183624,\n 'Loss/regularization_loss': 0.16143182,\n 'Loss/total_loss': 0.4877434,\n 'learning_rate': 0.07991781}\nI1230 20:10:13.916568 140299792009088 model_lib_v2.py:708] {'Loss/classification_loss': 0.18447536,\n 'Loss/localization_loss': 0.14183624,\n 'Loss/regularization_loss': 0.16143182,\n 'Loss/total_loss': 0.4877434,\n 'learning_rate': 0.07991781}","metadata":{}},{"cell_type":"markdown","source":"# Evaluate the Model","metadata":{"_uuid":"3c433991-1dfd-41e4-af69-ad2607eb5f10","_cell_guid":"098ff39e-b885-4ad6-ad0b-156256e854c8","id":"qXVmPp5xDigT","trusted":true}},{"cell_type":"code","source":"command = \"python {} --model_dir={} --pipeline_config_path={} --checkpoint_dir={}\".format(TRAINING_SCRIPT, CHECKPOINT_PATH, PIPELINE_CONFIG, CHECKPOINT_PATH)","metadata":{"_uuid":"1e1c49c3-e002-40e0-b31f-6bc84fd02f82","_cell_guid":"1206c16b-bf6a-4f83-bb8b-bd4844fb61b3","collapsed":false,"id":"mTs1ADQnDigU","jupyter":{"outputs_hidden":false},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"command","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.019\n\n Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.055\n \n Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.007\n \n Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.006\n \n Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.024\n \n Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.128\n \n Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.059\n \n Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.150\n \n Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.208\n \n Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.005\n \n Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.250\n \n Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.592\n \nINFO:tensorflow:Eval metrics at step 2000\nI1231 18:49:47.775108 139774748125056 model_lib_v2.py:1015] Eval metrics at step 2000\nINFO:tensorflow:\t+ DetectionBoxes_Precision/mAP: 0.019225\nI1231 18:49:47.794445 139774748125056 model_lib_v2.py:1018] \t+ DetectionBoxes_Precision/mAP: 0.019225\n\n\nINFO:tensorflow:\t+ DetectionBoxes_Precision/mAP@.50IOU: 0.055222\nI1231 18:49:47.796933 139774748125056 model_lib_v2.py:1018] \t+ DetectionBoxes_Precision/mAP@.50IOU: 0.055222\nINFO:tensorflow:\t+ DetectionBoxes_Precision/mAP@.75IOU: 0.006860\nI1231 18:49:47.799041 139774748125056 model_lib_v2.py:1018] \t+ DetectionBoxes_Precision/mAP@.75IOU: 0.006860\n\n\nINFO:tensorflow:\t+ DetectionBoxes_Precision/mAP (small): 0.005941\nI1231 18:49:47.801052 139774748125056 model_lib_v2.py:1018] \t+ DetectionBoxes_Precision/mAP (small): 0.005941\nINFO:tensorflow:\t+ DetectionBoxes_Precision/mAP (medium): 0.023949\nI1231 18:49:47.803217 139774748125056 model_lib_v2.py:1018] \t+ DetectionBoxes_Precision/mAP (medium): 0.023949\n\n\nINFO:tensorflow:\t+ DetectionBoxes_Precision/mAP (large): 0.127765\nI1231 18:49:47.805278 139774748125056 model_lib_v2.py:1018] \t+ DetectionBoxes_Precision/mAP (large): 0.127765\nINFO:tensorflow:\t+ DetectionBoxes_Recall/AR@1: 0.058537\nI1231 18:49:47.807327 139774748125056 model_lib_v2.py:1018] \t+ DetectionBoxes_Recall/AR@1: 0.058537\n\n\nINFO:tensorflow:\t+ DetectionBoxes_Recall/AR@10: 0.150000\nI1231 18:49:47.809374 139774748125056 model_lib_v2.py:1018] \t+ DetectionBoxes_Recall/AR@10: 0.150000\nINFO:tensorflow:\t+ DetectionBoxes_Recall/AR@100: 0.207520\nI1231 18:49:47.811417 139774748125056 model_lib_v2.py:1018] \t+ DetectionBoxes_Recall/AR@100: 0.207520\n\n\nINFO:tensorflow:\t+ DetectionBoxes_Recall/AR@100 (small): 0.004687\nI1231 18:49:47.813374 139774748125056 model_lib_v2.py:1018] \t+ DetectionBoxes_Recall/AR@100 (small): 0.004687\nINFO:tensorflow:\t+ DetectionBoxes_Recall/AR@100 (medium): 0.250448\nI1231 18:49:47.815356 139774748125056 model_lib_v2.py:1018] \t+ DetectionBoxes_Recall/AR@100 (medium): 0.250448\n\n\nINFO:tensorflow:\t+ DetectionBoxes_Recall/AR@100 (large): 0.592308\nI1231 18:49:47.817360 139774748125056 model_lib_v2.py:1018] \t+ DetectionBoxes_Recall/AR@100 (large): 0.592308\nINFO:tensorflow:\t+ Loss/localization_loss: 0.681640\nI1231 18:49:47.818885 139774748125056 model_lib_v2.py:1018] \t+ Loss/localization_loss: 0.681640\n\n\nINFO:tensorflow:\t+ Loss/classification_loss: 6.248461\nI1231 18:49:47.820402 139774748125056 model_lib_v2.py:1018] \t+ Loss/classification_loss: 6.248461\nINFO:tensorflow:\t+ Loss/regularization_loss: 0.161301\n\n\nI1231 18:49:47.821904 139774748125056 model_lib_v2.py:1018] \t+ Loss/regularization_loss: 0.161301\nINFO:tensorflow:\t+ Loss/total_loss: 7.091401\nI1231 18:49:47.823434 139774748125056 model_lib_v2.py:1018] \t+ Loss/total_loss: 7.091401","metadata":{}},{"cell_type":"code","source":"from IPython.display import Image\nImage(filename='../input/mobilenetresulttensorflowbeef/result/board1.png')","metadata":{"execution":{"iopub.status.busy":"2021-12-31T19:11:55.835521Z","iopub.execute_input":"2021-12-31T19:11:55.835866Z","iopub.status.idle":"2021-12-31T19:11:55.873359Z","shell.execute_reply.started":"2021-12-31T19:11:55.835777Z","shell.execute_reply":"2021-12-31T19:11:55.872545Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Image(filename='../input/mobilenetresulttensorflowbeef/result/board2.png')","metadata":{"execution":{"iopub.status.busy":"2021-12-31T19:12:10.888388Z","iopub.execute_input":"2021-12-31T19:12:10.888638Z","iopub.status.idle":"2021-12-31T19:12:10.913130Z","shell.execute_reply.started":"2021-12-31T19:12:10.888610Z","shell.execute_reply":"2021-12-31T19:12:10.912459Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Image(filename='../input/mobilenetresulttensorflowbeef/result/board3.png')","metadata":{"execution":{"iopub.status.busy":"2021-12-31T19:12:21.943579Z","iopub.execute_input":"2021-12-31T19:12:21.944247Z","iopub.status.idle":"2021-12-31T19:12:21.959093Z","shell.execute_reply.started":"2021-12-31T19:12:21.944210Z","shell.execute_reply":"2021-12-31T19:12:21.958485Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Image(filename='../input/mobilenetresulttensorflowbeef/result/board4.png')","metadata":{"execution":{"iopub.status.busy":"2021-12-31T19:12:28.467493Z","iopub.execute_input":"2021-12-31T19:12:28.468183Z","iopub.status.idle":"2021-12-31T19:12:28.497408Z","shell.execute_reply.started":"2021-12-31T19:12:28.468146Z","shell.execute_reply":"2021-12-31T19:12:28.496833Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Image(filename='../input/mobilenetresulttensorflowbeef/result/board5.png')","metadata":{"execution":{"iopub.status.busy":"2021-12-31T19:12:36.685291Z","iopub.execute_input":"2021-12-31T19:12:36.686025Z","iopub.status.idle":"2021-12-31T19:12:36.737284Z","shell.execute_reply.started":"2021-12-31T19:12:36.685983Z","shell.execute_reply":"2021-12-31T19:12:36.736731Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Image(filename='../input/mobilenetresulttensorflowbeef/result/board6.png')","metadata":{"execution":{"iopub.status.busy":"2021-12-31T19:12:49.599414Z","iopub.execute_input":"2021-12-31T19:12:49.599665Z","iopub.status.idle":"2021-12-31T19:12:49.648386Z","shell.execute_reply.started":"2021-12-31T19:12:49.599638Z","shell.execute_reply":"2021-12-31T19:12:49.647505Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from shutil import rmtree\nrmtree('./mobilenet')","metadata":{"execution":{"iopub.status.busy":"2021-12-31T19:35:36.983717Z","iopub.execute_input":"2021-12-31T19:35:36.983978Z","iopub.status.idle":"2021-12-31T19:35:37.214957Z","shell.execute_reply.started":"2021-12-31T19:35:36.983947Z","shell.execute_reply":"2021-12-31T19:35:37.213919Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now Lets see **YOLO V5**","metadata":{}},{"cell_type":"markdown","source":"![yolo](https://user-images.githubusercontent.com/26456083/86477109-5a7ca780-bd7a-11ea-9cb7-48d9fd6848e7.jpg)","metadata":{}},{"cell_type":"code","source":"# lets first download YOLO V5\n# Download YOLOv5\n!git clone https://github.com/ultralytics/yolov5  # clone repo\n%cd yolov5\n\n# Install dependencies\n%pip install -r requirements.txt  \n# change directory\n\n%cd ../\nimport torch\nprint(f\"Setup complete. Using torch {torch.__version__} ({torch.cuda.get_device_properties(0).name if torch.cuda.is_available() else 'CPU'})\")","metadata":{"execution":{"iopub.status.busy":"2021-12-31T19:49:58.452006Z","iopub.execute_input":"2021-12-31T19:49:58.452485Z","iopub.status.idle":"2021-12-31T19:50:12.254502Z","shell.execute_reply.started":"2021-12-31T19:49:58.452400Z","shell.execute_reply":"2021-12-31T19:50:12.253572Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install wandb\nimport wandb\n!wandb login 0e3d646e6d324479398418c8659583d46b1013ec","metadata":{"execution":{"iopub.status.busy":"2021-12-31T19:50:31.844077Z","iopub.execute_input":"2021-12-31T19:50:31.844989Z","iopub.status.idle":"2021-12-31T19:50:42.053158Z","shell.execute_reply.started":"2021-12-31T19:50:31.844932Z","shell.execute_reply":"2021-12-31T19:50:42.052093Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train['sequence'].unique()\n# The sequence represents ID of a gap-free subset of a given video\ndf_train['folds'] = -1","metadata":{"_uuid":"77b0f300-60e1-4ac2-88c8-0e755cde219e","_cell_guid":"73c1e03b-6ca2-4d64-a5d0-655d7ab5fbc3","collapsed":false,"id":"m1HQlJpHDigV","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:52:03.228979Z","iopub.execute_input":"2021-12-31T19:52:03.229285Z","iopub.status.idle":"2021-12-31T19:52:03.234988Z","shell.execute_reply.started":"2021-12-31T19:52:03.229252Z","shell.execute_reply":"2021-12-31T19:52:03.234179Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Lets split the dataset now.\nfrom sklearn.model_selection import GroupKFold\n\nsplit = GroupKFold(n_splits=4)\nX = df_train\nY = df_train.video_id.to_list()\ngroups = df_train.sequence.to_list()\nfor fold,(train_idx, val_idx) in enumerate(split.split(X,Y,groups)):\n    df_train.loc[val_idx, 'folds'] = fold\n\ndf_train['folds'].value_counts()","metadata":{"_uuid":"a5837bb9-3015-4194-801e-2bac5c6aa551","_cell_guid":"306a1013-ce99-4460-abb6-a5a0dd1092e3","collapsed":false,"id":"4lilmayfDigV","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:52:03.829335Z","iopub.execute_input":"2021-12-31T19:52:03.830179Z","iopub.status.idle":"2021-12-31T19:52:04.543870Z","shell.execute_reply.started":"2021-12-31T19:52:03.830132Z","shell.execute_reply":"2021-12-31T19:52:04.543160Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.sample(5)","metadata":{"_uuid":"397a4494-1a8f-4f7f-9265-0730059def16","_cell_guid":"edcc3197-c286-4ea2-aad5-ec4e4aa7c0fb","collapsed":false,"id":"5ZmbsOYkDigV","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:52:06.480390Z","iopub.execute_input":"2021-12-31T19:52:06.480867Z","iopub.status.idle":"2021-12-31T19:52:06.501846Z","shell.execute_reply.started":"2021-12-31T19:52:06.480829Z","shell.execute_reply":"2021-12-31T19:52:06.501183Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Some hyperparameters\nBATCH = 16\nIMG_SHAPE = 1280\nVAL_FOLD = 2\nEPOCH = 5","metadata":{"_uuid":"8a1ac4e6-a35c-4780-8bf8-0c3788899e4d","_cell_guid":"db106db6-1346-4376-934a-5b0cf72ac805","collapsed":false,"id":"KkxgP3JNDigV","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:52:11.536011Z","iopub.execute_input":"2021-12-31T19:52:11.536295Z","iopub.status.idle":"2021-12-31T19:52:11.541206Z","shell.execute_reply.started":"2021-12-31T19:52:11.536263Z","shell.execute_reply":"2021-12-31T19:52:11.539678Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Lets first create some directories and place our image there accordingly.\nos.makedirs('Starfish/images/train', exist_ok=True)\nos.makedirs('Starfish/images/val', exist_ok=True)\nos.makedirs('Starfish/labels/train', exist_ok=True)\nos.makedirs('Starfish/labels/val', exist_ok=True)","metadata":{"_uuid":"a7c3c934-9234-4e68-b7a2-9e2c4e4c4b0d","_cell_guid":"1f080117-b991-4dd7-b3b8-0ae8c3b3108a","collapsed":false,"id":"FLYzLUA6DigV","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:52:16.377453Z","iopub.execute_input":"2021-12-31T19:52:16.378405Z","iopub.status.idle":"2021-12-31T19:52:16.384172Z","shell.execute_reply.started":"2021-12-31T19:52:16.378354Z","shell.execute_reply":"2021-12-31T19:52:16.383488Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for idx in tqdm(range(df_train.shape[0])):\n    instance = df_train.iloc[idx]\n    if instance[\"folds\"] != VAL_FOLD:\n        copyfile(f'{instance[\"path\"]}.jpg', 'Starfish/images/train/{}.jpg'.format(instance[\"image_id\"]))\n    else:\n        copyfile(f'{instance[\"path\"]}.jpg', 'Starfish/images/val/{}.jpg'.format(instance[\"image_id\"]))","metadata":{"_uuid":"f74c1159-2285-4744-95e2-450ac7da6193","_cell_guid":"f8480133-7f5c-4e75-8666-21a39c326be6","collapsed":false,"id":"6LlVq2ZVDigW","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:52:18.601985Z","iopub.execute_input":"2021-12-31T19:52:18.602276Z","iopub.status.idle":"2021-12-31T19:53:10.230923Z","shell.execute_reply.started":"2021-12-31T19:52:18.602239Z","shell.execute_reply":"2021-12-31T19:53:10.230177Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Lets Analayse the instances\nTRAIN_PATH = '/kaggle/working/Starfish/images/train'\nVAL_PATH = '/kaggle/working/Starfish/images/val'\n\nprint('Number of training instances are: {}'.format(len(os.listdir(TRAIN_PATH))))\nprint('Number of validation instances are: {}'.format(len(os.listdir(VAL_PATH))))","metadata":{"_uuid":"79f3d5ca-bba4-4238-9b0a-3276db26372d","_cell_guid":"71c0cba2-77b3-450a-ab83-b5155f8da60c","collapsed":false,"id":"Xyg9tleaDigW","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:53:16.532999Z","iopub.execute_input":"2021-12-31T19:53:16.533273Z","iopub.status.idle":"2021-12-31T19:53:16.544061Z","shell.execute_reply.started":"2021-12-31T19:53:16.533237Z","shell.execute_reply":"2021-12-31T19:53:16.543169Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Next step is to create yaml file which is the configuration file that YOLO V5 will read\nIt contains three minimal things:\n\n    1. Path to the train/val/test images we can give in .txt format.\n    \n    2. Number of classes\n    \n    3. List of the names of classes","metadata":{"_uuid":"0167bd66-1ef0-47a2-adac-f48399e38e77","_cell_guid":"c5517e45-310a-4760-aa1a-c57b06a3d2ec","id":"OrZuGHD3DigX","trusted":true}},{"cell_type":"code","source":"import yaml,glob\n\nwith open('/kaggle/working/train.txt', 'w') as work_train:\n    for path in glob.glob(f\"{TRAIN_PATH}/*\"):\n        work_train.write(path+'\\n')\n    \nwith open('/kaggle/working/val.txt', 'w') as work_val:\n    for path in glob.glob(f\"{VAL_PATH}/*\"):\n        work_val.write(path+'\\n')","metadata":{"_uuid":"aafab1d9-7a53-41cb-a29b-1e837a3fe13f","_cell_guid":"645ea85d-c39d-4663-9cb8-dbff4e853f59","collapsed":false,"id":"2uV1vUzuDigX","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:53:49.646494Z","iopub.execute_input":"2021-12-31T19:53:49.647180Z","iopub.status.idle":"2021-12-31T19:53:49.672796Z","shell.execute_reply.started":"2021-12-31T19:53:49.647141Z","shell.execute_reply":"2021-12-31T19:53:49.672178Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"config = dict(   # from yolo github repo\n    train = TRAIN_PATH,\n    val = VAL_PATH,\n    nc = 1,  #classes\n    names = ['COTS']\n)","metadata":{"_uuid":"f4a7312b-d917-4c21-a5f2-e3846d033210","_cell_guid":"f3434cdc-ce2c-43b6-8923-f5c8e1084885","collapsed":false,"id":"OiYElEdlDigY","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:53:54.546206Z","iopub.execute_input":"2021-12-31T19:53:54.546486Z","iopub.status.idle":"2021-12-31T19:53:54.550387Z","shell.execute_reply.started":"2021-12-31T19:53:54.546458Z","shell.execute_reply":"2021-12-31T19:53:54.549630Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with open(\"./yolov5/data/data.yaml\",'w') as configuration:\n    yaml.dump(config, configuration, default_flow_style=False)\n\n# If you want collections to be always serialized in the block style,\n#    set the parameter default_flow_style of dump() to False","metadata":{"_uuid":"28bd62a6-cc5b-4d7d-a594-ef740564b879","_cell_guid":"ce1008b4-2e5a-452e-9320-b6ccfa9cbd8d","collapsed":false,"id":"s02TuRhsDigY","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:54:16.103134Z","iopub.execute_input":"2021-12-31T19:54:16.103761Z","iopub.status.idle":"2021-12-31T19:54:16.109175Z","shell.execute_reply.started":"2021-12-31T19:54:16.103721Z","shell.execute_reply":"2021-12-31T19:54:16.108430Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.listdir(\"./yolov5/data\")","metadata":{"_uuid":"381f02a7-2a2e-4028-811f-6b81bec1e570","_cell_guid":"df7b7005-f263-4e07-b24f-939b35ea79bc","collapsed":false,"id":"sLoLQDUQDigY","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:54:19.747296Z","iopub.execute_input":"2021-12-31T19:54:19.748310Z","iopub.status.idle":"2021-12-31T19:54:19.756167Z","shell.execute_reply.started":"2021-12-31T19:54:19.748259Z","shell.execute_reply":"2021-12-31T19:54:19.755276Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Create Labels for YOLOv5\nTo label your images,a .txt file with the same name of the image,will be created (if no objects in image, no *.txt file is required) The *.txt file specifications are:\n\nOne row per object\n\nEach row is class x_center y_center width height format.\n\nBox coordinates must be in normalized xywh format (from 0 - 1). If your boxes are in pixels, divide x_center and width by image width, and y_center and height by image height.\n\nClass numbers are zero-indexed (start from 0).\n\n\n📍 Note: We don't have to remove the images without bounding boxes from the training or validation sets.","metadata":{"_uuid":"d6768178-5c7d-48ed-b9b7-985ffa76d822","_cell_guid":"cfb07bb1-496d-430a-843b-ad9c5afb4979","id":"mvYaeAyGDigY","trusted":true}},{"cell_type":"code","source":"df_train.sample(2)","metadata":{"_uuid":"b9478817-f77e-4d9a-8402-de40de3323af","_cell_guid":"cfd75732-bb34-4503-925e-58f433487782","collapsed":false,"id":"L5LZf27BDigZ","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:54:23.933590Z","iopub.execute_input":"2021-12-31T19:54:23.933857Z","iopub.status.idle":"2021-12-31T19:54:23.948501Z","shell.execute_reply.started":"2021-12-31T19:54:23.933828Z","shell.execute_reply":"2021-12-31T19:54:23.947657Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"All_boxes = []\nfor idx in tqdm(range(df_train.shape[0])):\n    instance = df_train.iloc[idx]\n    \n    image_id = instance['image_id']\n    height = 720\n    width = 1280\n    boxes = np.array(instance['box_location']).astype(np.float).copy()\n    num_boxes = len(boxes)\n    names = ['COTS'] * num_boxes\n    labels = [0] * num_boxes\n    if instance[\"folds\"] != VAL_FOLD:\n        filename = '/kaggle/working/Starfish/labels/train/{}.txt'.format(instance[\"image_id\"])\n    else:\n        filename = '/kaggle/working/Starfish/labels/val/{}.txt'.format(instance[\"image_id\"])\n    \n    with open(filename, 'w') as file:\n        normalized_boxes = normalize_image(boxes)\n        cliped_boxes = np.clip(normalized_boxes, 0, 1)\n        All_boxes.extend(cliped_boxes)\n        for box_idx in range(len(cliped_boxes)):\n            bb = str(cliped_boxes[box_idx])[1:-1]   \n            annot = str(str(labels[box_idx])) + ' ' + bb + '\\n'\n            annot = ''.join(annot)\n            annot = annot.strip('')\n            file.write(annot)","metadata":{"_uuid":"26c49134-07f8-4b85-970a-0a9e266cda70","_cell_guid":"86d3a4ba-643e-49cf-85aa-b175e07c7573","collapsed":false,"id":"1m4DaFtIDigZ","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:54:26.478799Z","iopub.execute_input":"2021-12-31T19:54:26.479404Z","iopub.status.idle":"2021-12-31T19:54:30.279570Z","shell.execute_reply.started":"2021-12-31T19:54:26.479364Z","shell.execute_reply":"2021-12-31T19:54:30.278606Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_instances = len(os.listdir('/kaggle/working/Starfish/labels/train'))\nval_instances = len(os.listdir('/kaggle/working/Starfish/labels/val'))\nprint('training instances are: {}'.format(train_instances))\nprint('validation instances are: {}'.format(val_instances))","metadata":{"_uuid":"786a3864-9e1f-4327-a883-901224d57a5a","_cell_guid":"06459641-d2a9-476f-bd21-863ea849201a","collapsed":false,"id":"GZwJK7W_DigZ","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:54:43.494120Z","iopub.execute_input":"2021-12-31T19:54:43.494858Z","iopub.status.idle":"2021-12-31T19:54:43.504561Z","shell.execute_reply.started":"2021-12-31T19:54:43.494817Z","shell.execute_reply":"2021-12-31T19:54:43.503646Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Training with W&B**","metadata":{"_uuid":"82bb2c95-b284-4dcf-a95b-f5dd0ad4ddc9","_cell_guid":"545b7ce3-156c-43bc-b2be-e1395335e0d5","id":"_TbBXZ_WDiga","trusted":true}},{"cell_type":"code","source":"%cd yolov5/","metadata":{"_uuid":"2853901a-e021-49d5-aa03-29dc310f9e9b","_cell_guid":"9eb52cea-f7ab-4294-964f-4e50297580ca","collapsed":false,"id":"UPeW4WC9Diga","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:54:46.404001Z","iopub.execute_input":"2021-12-31T19:54:46.404579Z","iopub.status.idle":"2021-12-31T19:54:46.411395Z","shell.execute_reply.started":"2021-12-31T19:54:46.404538Z","shell.execute_reply":"2021-12-31T19:54:46.410573Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch\ntorch.cuda.empty_cache()","metadata":{"_uuid":"5bf403cb-79d8-4ab3-9ec7-3a705273bff1","_cell_guid":"c87bdf85-814d-41d1-93a4-dd7edbd50fa1","collapsed":false,"id":"VMUc1Qw1Diga","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:54:47.757525Z","iopub.execute_input":"2021-12-31T19:54:47.758084Z","iopub.status.idle":"2021-12-31T19:54:47.761819Z","shell.execute_reply.started":"2021-12-31T19:54:47.758047Z","shell.execute_reply":"2021-12-31T19:54:47.761119Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!python train.py --img {IMG_SHAPE} \\\n                 --batch {BATCH} \\\n                 --epochs {EPOCH} \\\n                 --data data.yaml \\\n                 --weights yolov5s.pt \\\n                 --project tf-reef","metadata":{"_uuid":"88abdb58-befc-4c17-80e1-a057b0f69523","_cell_guid":"9a4b2ae9-0c7e-4c73-bfc5-0381fa4fd3e7","collapsed":false,"id":"MYPrzFSTDiga","jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-12-31T19:54:48.918023Z","iopub.execute_input":"2021-12-31T19:54:48.918600Z","iopub.status.idle":"2021-12-31T20:39:13.787306Z","shell.execute_reply.started":"2021-12-31T19:54:48.918563Z","shell.execute_reply":"2021-12-31T20:39:13.786455Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.listdir(\"./kaggle/yolov5\")","metadata":{"_uuid":"85826c97-7736-4429-8cbd-d9b7dceade70","_cell_guid":"c7b986f1-c2d1-4e04-9d4a-96b8c25ab1ca","collapsed":false,"id":"uiq4HG1nDiga","jupyter":{"outputs_hidden":false},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"_uuid":"4cfce4ce-2227-4a0f-ab6d-4a8b53404ef0","_cell_guid":"f2403815-f75d-491f-b9f3-b8f5d4301e85","collapsed":false,"id":"ZvINRYz4Digb","jupyter":{"outputs_hidden":false},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![work in progress](https://simpro.it/wp-content/uploads/2015/07/Work-in-progress-1024x603.png)","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}