{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"#### What are you trying to do in this notebook?\nMy goal for this competition is to accurate identify starfish in real-time by building an object detection model trained on underwater videos of coral reefs. My work will help researchers to identify species that are threatening Australia's Great Barrier Reef and take well-informed action to protect the reef for future generations. In this notebook we explore sequences as potential units for cross-validation, but since there are only 20 sequences and their sizes are quite disimilar, we propose an approach to split them into smaller chunks, that we name subsequences.\n\n#### Why are you trying it?\nTo detect crown-of-thorns starfish in underwater image data. In this competition, I will predict the presence and position of crown-of-thorns starfish in sequences of underwater images taken at various times and locations around the Great Barrier Reef. Predictions take the form of a bounding box together with a confidence score for each identified starfish. An image may contain zero or more starfish.\n\nIn this notebook we explore sequences as potential units for cross-validation, but since there are only 20 sequences and their sizes are quite disimilar, we propose an approach to split them into smaller chunks, that we name subsequences.\n\nA sequence, as stated in the data tab of the competition, is:\n\nsequence - ID of a gap-free subset of a given video. The sequence ids are not meaningfully ordered.\n\nSubsequences, as we will define them below, are parts of a sequences where objects are continually present or are continually not present. We isolate 2 kind of subsequences: with objects and with no objects.\n\n\n##### This notebook demonstrates how to run inference using an EfficientDet-D0 model trained with TensorFlow Object Detection API, and submit the detection result. See [this notebook](https://www.kaggle.com/khanhlvg/cots-detection-w-tensorflow-object-detection-api/) for details on how the model was trained.","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19"}},{"cell_type":"code","source":"import numpy as np\nimport os\nimport sys\nimport tensorflow as tf\nimport time\n\n# Import the library that is used to submit the prediction result.\nINPUT_DIR = '../input/tensorflow-great-barrier-reef/'\nsys.path.insert(0, INPUT_DIR)\nimport greatbarrierreef","metadata":{"execution":{"iopub.status.busy":"2021-12-08T11:53:09.487298Z","iopub.execute_input":"2021-12-08T11:53:09.487625Z","iopub.status.idle":"2021-12-08T11:53:13.658740Z","shell.execute_reply.started":"2021-12-08T11:53:09.487548Z","shell.execute_reply":"2021-12-08T11:53:13.657923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Load the TensorFlow COTS detection model into memory and define some util functions for running inference.","metadata":{"execution":{"iopub.status.busy":"2021-11-23T03:32:24.29052Z","iopub.execute_input":"2021-11-23T03:32:24.290814Z","iopub.status.idle":"2021-11-23T03:32:24.295257Z","shell.execute_reply.started":"2021-11-23T03:32:24.290785Z","shell.execute_reply":"2021-11-23T03:32:24.294012Z"}}},{"cell_type":"code","source":"MODEL_DIR = '../input/cots-detection-w-tensorflow-object-detection-api/cots_efficientdet_d0'\nstart_time = time.time()\ntf.keras.backend.clear_session()\ndetect_fn_tf_odt = tf.saved_model.load(os.path.join(os.path.join(MODEL_DIR, 'output'), 'saved_model'))\nend_time = time.time()\nelapsed_time = end_time - start_time\nprint('Elapsed time: ' + str(elapsed_time) + 's')","metadata":{"execution":{"iopub.status.busy":"2021-12-08T11:53:13.660226Z","iopub.execute_input":"2021-12-08T11:53:13.661945Z","iopub.status.idle":"2021-12-08T11:53:43.663375Z","shell.execute_reply.started":"2021-12-08T11:53:13.661915Z","shell.execute_reply":"2021-12-08T11:53:43.662357Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def load_image_into_numpy_array(path):\n    \"\"\"Load an image from file into a numpy array.\n\n    Puts image into numpy array to feed into tensorflow graph.\n    Note that by convention we put it into a numpy array with shape\n    (height, width, channels), where channels=3 for RGB.\n\n    Args:\n    path: a file path (this can be local or on colossus)\n\n    Returns:\n    uint8 numpy array with shape (img_height, img_width, 3)\n    \"\"\"\n    img_data = tf.io.gfile.GFile(path, 'rb').read()\n    image = Image.open(io.BytesIO(img_data))\n    (im_width, im_height) = image.size\n    \n    return np.array(image.getdata()).reshape(\n      (im_height, im_width, 3)).astype(np.uint8)\n\ndef detect(image_np):\n    \"\"\"Detect COTS from a given numpy image.\"\"\"\n\n    input_tensor = np.expand_dims(image_np, 0)\n    start_time = time.time()\n    detections = detect_fn_tf_odt(input_tensor)\n    return detections","metadata":{"execution":{"iopub.status.busy":"2021-12-08T11:53:43.667410Z","iopub.execute_input":"2021-12-08T11:53:43.667650Z","iopub.status.idle":"2021-12-08T11:53:43.686141Z","shell.execute_reply.started":"2021-12-08T11:53:43.667619Z","shell.execute_reply":"2021-12-08T11:53:43.682523Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Run inference and create the submission data","metadata":{}},{"cell_type":"code","source":"env = greatbarrierreef.make_env()   # initialize the environment\niter_test = env.iter_test()    # an iterator which loops over the test set and sample submission","metadata":{"execution":{"iopub.status.busy":"2021-12-08T11:53:43.690680Z","iopub.execute_input":"2021-12-08T11:53:43.691074Z","iopub.status.idle":"2021-12-08T11:53:43.702948Z","shell.execute_reply.started":"2021-12-08T11:53:43.691033Z","shell.execute_reply":"2021-12-08T11:53:43.702056Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"DETECTION_THRESHOLD = 0.25\n\nsubmission_dict = {\n    'id': [],\n    'prediction_string': [],\n}\n\nfor (image_np, sample_prediction_df) in iter_test:\n    height, width, _ = image_np.shape\n    \n    # Run object detection using the TensorFlow model.\n    detections = detect(image_np)\n    \n    # Parse the detection result and generate a prediction string.\n    num_detections = detections['num_detections'][0].numpy().astype(np.int32)\n    predictions = []\n    for index in range(num_detections):\n        score = detections['detection_scores'][0][index].numpy()\n        if score < DETECTION_THRESHOLD:\n            continue\n\n        bbox = detections['detection_boxes'][0][index].numpy()\n        y_min = int(bbox[0] * height)\n        x_min = int(bbox[1] * width)\n        y_max = int(bbox[2] * height)\n        x_max = int(bbox[3] * width)\n        \n        bbox_width = x_max - x_min\n        bbox_height = y_max - y_min\n        \n        predictions.append('{:.2f} {} {} {} {}'.format(score, x_min, y_min, bbox_width, bbox_height))\n    \n    # Generate the submission data.\n    prediction_str = ' '.join(predictions)\n    sample_prediction_df['annotations'] = prediction_str\n    env.predict(sample_prediction_df)\n\n    print('Prediction:', prediction_str)","metadata":{"execution":{"iopub.status.busy":"2021-12-08T11:53:43.706345Z","iopub.execute_input":"2021-12-08T11:53:43.706628Z","iopub.status.idle":"2021-12-08T11:53:52.630695Z","shell.execute_reply.started":"2021-12-08T11:53:43.706592Z","shell.execute_reply":"2021-12-08T11:53:52.629949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Did it work?\nThis competition uses a hidden test set that will be served by an API to ensure you evaluate the images in the same order they were recorded within each video. I tried doing EDA for the provided dataset. I plan to dig much dipper and understand the data in a much better way. There are alot of things to discover from this dataset.\n\n#### What did you not understand about this process?\nWell, everything provides in the competition data page. I've no problem while working on it. If you guys don't understand the thing that I'll do in this notebook then please comment on this notebook.\n\n#### What else do you think you can try as part of this approach?\nWe solve the greatest challenges through innovative science and technology to unlock a better future for everyone. We are thinkers, problem solvers, leaders. We blaze new trails of discovery. We aim to inspire the next generation. The Great Barrier Reef Foundation creates a better future for coral reefs and their marine life through innovative projects and global advocacy efforts.\n\n#### PLEASE UPVOTE if you find this notebook is useful for you !","metadata":{}}]}