{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# TensorFlow - Help Protect the Great Barrier Reef\n\n# Table of Content\n1. Introduction\n2. Load Dataset & EDA\n3. Bounding Box Analysis\n 1. Ratio between Number of Boxes with Objects and Number of Boxes without Objects\n 2. How to decode annotations from the dataframe loaded from the train.csv file\n 3. Summary of Number of Boxes in Each Frame and Distributions of Number of Boxes\n 4. Bounding Box Visualization\n 5. Distribution of Bounding Box Center Coordinates on Image\n4. Sequence Preview\n 1. Generate Annotated Video from Sequences\n\n# Introduction\n\n

In this competition, our goal is to predict the presence and position of crown-of-thorns starfish in sequences of underwater images taken at various times and locations around the Great Barrier Reef. Predictions take the form of a bounding box together with a confidence score for each identified starfish. An image may contain zero or more starfish. Our model should evaluate the images in the same order as they were recorded in the video.

\n\n\n\n[Data Metadata](https://www.kaggle.com/c/tensorflow-great-barrier-reef/data): Based on the description in the data's metadata, we can summarize the structure of the data as following:\n1. image_id is in a format of: video_id + \"-\" + video_frame\n2. bounding box format is: (xmin, ymin, width, height) in pixels, the annotation is a list of dictionary with structure {x, y, width, height}\n3. there are gaps between frames in videos\n4. sequence is a subset of frames without gaps, but with no ordering\n\n

If you have any suggestions or find any issues related to interpretation, welcome to comment and to point those out. Thank you! Hope this notebook can be helpful for getting a better understanding about the data.

","metadata":{}},{"cell_type":"code","source":"import os\n\nfrom PIL import Image, ImageDraw\nimport cv2\nimport re\nimport pandas as pd\nimport numpy as np\nfrom tqdm import tqdm\n\nfrom matplotlib import pyplot as plt\nimport matplotlib.patches as patches\nimport seaborn as sns\nfrom IPython.display import Video, display","metadata":{"execution":{"iopub.status.busy":"2021-11-25T04:36:12.859477Z","iopub.execute_input":"2021-11-25T04:36:12.859731Z","iopub.status.idle":"2021-11-25T04:36:12.865427Z","shell.execute_reply.started":"2021-11-25T04:36:12.859707Z","shell.execute_reply":"2021-11-25T04:36:12.864390Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load Dataset & EDA","metadata":{}},{"cell_type":"code","source":"dataset = {\n 'root_dir': '../input/tensorflow-great-barrier-reef',\n 'train_csv': '../input/tensorflow-great-barrier-reef/train.csv',\n 'test_csv': '../input/tensorflow-great-barrier-reef/test.csv',\n 'sample_submission_csv': '../input/tensorflow-great-barrier-reef/example_sample_submission.csv',\n 'video_img_dir': '../input/tensorflow-great-barrier-reef/train_images'\n}","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:42:10.203385Z","iopub.execute_input":"2021-11-25T03:42:10.203602Z","iopub.status.idle":"2021-11-25T03:42:10.208219Z","shell.execute_reply.started":"2021-11-25T03:42:10.203580Z","shell.execute_reply":"2021-11-25T03:42:10.207060Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_csv = pd.read_csv(dataset['train_csv'])\ntest_csv = pd.read_csv(dataset['test_csv'])","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:42:10.619528Z","iopub.execute_input":"2021-11-25T03:42:10.619807Z","iopub.status.idle":"2021-11-25T03:42:10.677937Z","shell.execute_reply.started":"2021-11-25T03:42:10.619779Z","shell.execute_reply":"2021-11-25T03:42:10.677357Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"number of frames:\", len(train_csv))","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:51:48.360216Z","iopub.execute_input":"2021-11-25T03:51:48.360646Z","iopub.status.idle":"2021-11-25T03:51:48.366433Z","shell.execute_reply.started":"2021-11-25T03:51:48.360592Z","shell.execute_reply":"2021-11-25T03:51:48.365363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_csv.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:42:11.325149Z","iopub.execute_input":"2021-11-25T03:42:11.325407Z","iopub.status.idle":"2021-11-25T03:42:11.347865Z","shell.execute_reply.started":"2021-11-25T03:42:11.325381Z","shell.execute_reply":"2021-11-25T03:42:11.346878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Let's first see how many frames are there for each video; also, we need to notice that the frames might not be consecutive (there might be gaps in each video

","metadata":{}},{"cell_type":"code","source":"frame_counts = train_csv['video_id'].value_counts().sort_values().to_frame()\nframe_counts.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:43:04.516437Z","iopub.execute_input":"2021-11-25T03:43:04.516849Z","iopub.status.idle":"2021-11-25T03:43:04.531464Z","shell.execute_reply.started":"2021-11-25T03:43:04.516823Z","shell.execute_reply":"2021-11-25T03:43:04.531066Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Then, let's run a sanity check to make sure the number of images in the data folder match with the number of images in the train file

","metadata":{}},{"cell_type":"code","source":"print(\"number of records in video_0 matched: \", frame_counts.loc[0]['video_id'] == len(os.listdir(os.path.join(dataset['video_img_dir'], 'video_0'))))\nprint(\"number of records in video_0 matched: \", frame_counts.loc[1]['video_id'] == len(os.listdir(os.path.join(dataset['video_img_dir'], 'video_1'))))\nprint(\"number of records in video_0 matched: \", frame_counts.loc[2]['video_id'] == len(os.listdir(os.path.join(dataset['video_img_dir'], 'video_2'))))","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:46:23.409583Z","iopub.execute_input":"2021-11-25T03:46:23.409855Z","iopub.status.idle":"2021-11-25T03:46:23.966076Z","shell.execute_reply.started":"2021-11-25T03:46:23.409829Z","shell.execute_reply":"2021-11-25T03:46:23.964917Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

There are 20 distinct sequences in the dataset, and a sequence is a gap-free subset of a given video, and we calculate number of frames in each sequence

","metadata":{}},{"cell_type":"code","source":"sequence_counts = train_csv['sequence'].value_counts().sort_values().reset_index()\nsequence_counts.columns = [['sequence', 'num_frames']]\nprint(\"number of sequences:\", len(sequence_counts))\nsequence_counts.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:51:19.677031Z","iopub.execute_input":"2021-11-25T03:51:19.677886Z","iopub.status.idle":"2021-11-25T03:51:19.698353Z","shell.execute_reply.started":"2021-11-25T03:51:19.677859Z","shell.execute_reply":"2021-11-25T03:51:19.697645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Bounding Box Analysis","metadata":{}},{"cell_type":"code","source":"num_no_obj_frame = train_csv[train_csv.annotations == '[]']['annotations'].count()\nprint(\"number of frames without objects:\", num_no_obj_frame)","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:52:09.958232Z","iopub.execute_input":"2021-11-25T03:52:09.958689Z","iopub.status.idle":"2021-11-25T03:52:09.972862Z","shell.execute_reply.started":"2021-11-25T03:52:09.958658Z","shell.execute_reply":"2021-11-25T03:52:09.972134Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"num_with_obj_frame = train_csv[train_csv.annotations != '[]']['annotations'].count()\nprint(\"number of frames with objects:\", num_with_obj_frame)","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:52:10.607867Z","iopub.execute_input":"2021-11-25T03:52:10.608273Z","iopub.status.idle":"2021-11-25T03:52:10.622855Z","shell.execute_reply.started":"2021-11-25T03:52:10.608236Z","shell.execute_reply":"2021-11-25T03:52:10.621522Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

From above, we can see that the number of frames with no objects is almost 3.7 times of the number of frames with objects, and only 21% of the frames in the training data contains objects

","metadata":{}},{"cell_type":"code","source":"train_csv[train_csv.annotations != '[]'].head()","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:52:12.378652Z","iopub.execute_input":"2021-11-25T03:52:12.379266Z","iopub.status.idle":"2021-11-25T03:52:12.400283Z","shell.execute_reply.started":"2021-11-25T03:52:12.379233Z","shell.execute_reply":"2021-11-25T03:52:12.399314Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Ratio between Frames with Objects and Frames with No Objects","metadata":{}},{"cell_type":"code","source":"print('ratio of frames with objects:', num_with_obj_frame / len(train_csv))\n\nfig, axes = plt.subplots(1,1, figsize=(12, 6))\n\nsns.barplot(ax=axes, x=['Number of Frames with Objects', 'Number of Frames with No Objects'], y=[num_with_obj_frame, num_no_obj_frame])\naxes.set_title(\"Distribution of Frames with/without Objects\")\naxes.set_xlabel(\"Frame Types\")\naxes.set_ylabel(\"Count\")\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:54:37.473187Z","iopub.execute_input":"2021-11-25T03:54:37.473440Z","iopub.status.idle":"2021-11-25T03:54:37.677938Z","shell.execute_reply.started":"2021-11-25T03:54:37.473407Z","shell.execute_reply":"2021-11-25T03:54:37.676195Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Let's construct a function that can read the annotations and convert it to the format that can be used in object detection algorithm, and we calculate the number of boxes as well to help us get better understanding about the bounding box information

\n","metadata":{}},{"cell_type":"markdown","source":"## Decode Annotations","metadata":{}},{"cell_type":"code","source":"def decode_annotation(annot_line):\n # annot_line example: [{'x': 540, 'y': 310, 'width': 113, 'height': 105}, {'x': 657, 'y': 501, 'width': 95, 'height': 56}]\n boxes = []\n \n box_pattern = r'\\{\\'\\w\\'\\:\\s\\d+\\,\\s\\'\\w\\'\\:\\s\\d+\\,\\s\\'\\w+\\'\\:\\s\\d+\\,\\s\\'\\w+\\'\\:\\s\\d+\\}'\n val_pattern = r'\\d+'\n \n annotations = re.findall(box_pattern, annot_line)\n for annot in annotations:\n x, y, width, height = re.findall(val_pattern, annot)\n x, y, width, height = float(x), float(y), float(width), float(height)\n confidence = 1.0\n \n box = [x, y, width, height, confidence]\n boxes.append(box)\n \n return boxes\n\ndef count_boxes(annot_line):\n \n annot_line = annot_line[1:-1]\n box_pattern = r'\\{\\'\\w\\'\\:\\s\\d+\\,\\s\\'\\w\\'\\:\\s\\d+\\,\\s\\'\\w+\\'\\:\\s\\d+\\,\\s\\'\\w+\\'\\:\\s\\d+\\}'\n val_pattern = r'\\d+'\n \n annotations = re.findall(box_pattern, annot_line)\n \n return len(annotations)\n\n\ndef test_decode_annotation(annot_line):\n print(\"sample:\", annot_line)\n boxes = decode_annotation(annot_line)\n for i, box in enumerate(boxes):\n print(f\"box {i}:\", box)\n","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:55:14.695229Z","iopub.execute_input":"2021-11-25T03:55:14.696032Z","iopub.status.idle":"2021-11-25T03:55:14.704310Z","shell.execute_reply.started":"2021-11-25T03:55:14.696002Z","shell.execute_reply":"2021-11-25T03:55:14.703635Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_samples = [\n \"[{'x': 540, 'y': 310, 'width': 113, 'height': 105}, {'x': 657, 'y': 501, 'width': 95, 'height': 56}, {'x': 257, 'y': 101, 'width': 42, 'height': 59}]\",\n \"[{'x': 540, 'y': 310, 'width': 113, 'height': 105}, {'x': 657, 'y': 501, 'width': 95, 'height': 59}]\",\n \"[{'x': 12, 'y': 250, 'width': 143, 'height': 82}]\",\n \"[]\"\n]\n\nfor i, sample in enumerate(test_samples):\n num_boxes = count_boxes(sample)\n print(f\"Test {i+1}:\", f\"found {num_boxes} boxes\")\n \n test_decode_annotation(sample)\n print(\"\")","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:55:19.359061Z","iopub.execute_input":"2021-11-25T03:55:19.359304Z","iopub.status.idle":"2021-11-25T03:55:19.367902Z","shell.execute_reply.started":"2021-11-25T03:55:19.359273Z","shell.execute_reply":"2021-11-25T03:55:19.366923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Number of Boxes for each Frame","metadata":{}},{"cell_type":"code","source":"train_csv['num_boxes'] = train_csv['annotations'].apply(count_boxes)","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:55:22.578568Z","iopub.execute_input":"2021-11-25T03:55:22.578932Z","iopub.status.idle":"2021-11-25T03:55:22.623357Z","shell.execute_reply.started":"2021-11-25T03:55:22.578905Z","shell.execute_reply":"2021-11-25T03:55:22.622229Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_csv[train_csv.annotations != '[]'].head()","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:55:29.390517Z","iopub.execute_input":"2021-11-25T03:55:29.390773Z","iopub.status.idle":"2021-11-25T03:55:29.405298Z","shell.execute_reply.started":"2021-11-25T03:55:29.390750Z","shell.execute_reply":"2021-11-25T03:55:29.404663Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

We can observe that most of the frames have only one bounding box, and 3 frames contain 18 bounding boxes

","metadata":{}},{"cell_type":"code","source":"boxes_dist = train_csv[train_csv.annotations != '[]']['num_boxes'].value_counts().sort_values(ascending=False).reset_index()\nboxes_dist.columns = ['num_boxes', 'num_frames']\nboxes_dist","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:57:09.231815Z","iopub.execute_input":"2021-11-25T03:57:09.232128Z","iopub.status.idle":"2021-11-25T03:57:09.244259Z","shell.execute_reply.started":"2021-11-25T03:57:09.232104Z","shell.execute_reply":"2021-11-25T03:57:09.243043Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Distribution of Number of Boxes (Frame Counts)","metadata":{}},{"cell_type":"code","source":"fig = plt.figure(figsize=(24, 8))\nsns.barplot(x=boxes_dist.num_boxes, y=boxes_dist.num_frames)\n\nplt.title(\"Box Distribution\")\nplt.xlabel(\"Number of Boxes\")\nplt.ylabel(\"Frame Counts\")\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-25T03:58:26.558664Z","iopub.execute_input":"2021-11-25T03:58:26.558995Z","iopub.status.idle":"2021-11-25T03:58:26.838047Z","shell.execute_reply.started":"2021-11-25T03:58:26.558973Z","shell.execute_reply":"2021-11-25T03:58:26.837655Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Let's visualize the image and draw the bounding boxes on it, for each case mentioned in the last step, to see some sample data

","metadata":{}},{"cell_type":"markdown","source":"## Bounding Box Visualization","metadata":{}},{"cell_type":"code","source":"def gen_file_path(image_id):\n # extract file path by using the image_id in the train file\n video_id = image_id.split('-')[0]\n image_id = image_id.split('-')[1]\n return os.path.join(dataset['video_img_dir'], 'video_' + video_id, image_id + '.jpg')\n\ndef draw_boxes(image_path, annot_line):\n \n boxes = decode_annotation(annot_line)\n\n coords = [] \n for box in boxes: \n coord = [] \n coord.append(box[0]) \n coord.append(box[1]) \n coord.append(box[0] + box[2]) \n coord.append(box[1] + box[3]) \n coords.append(coord) \n\n image = Image.open(image_path)\n imgcp = image.copy()\n imgcp_draw = ImageDraw.Draw(imgcp)\n\n for coord in coords:\n imgcp_draw.rectangle(coord, fill = None, outline = \"red\", width=5)\n\n return imgcp","metadata":{"execution":{"iopub.status.busy":"2021-11-25T04:00:03.406736Z","iopub.execute_input":"2021-11-25T04:00:03.406989Z","iopub.status.idle":"2021-11-25T04:00:03.412314Z","shell.execute_reply.started":"2021-11-25T04:00:03.406966Z","shell.execute_reply":"2021-11-25T04:00:03.411385Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_csv['file_path'] = train_csv['image_id'].apply(gen_file_path)","metadata":{"execution":{"iopub.status.busy":"2021-11-25T04:00:04.518264Z","iopub.execute_input":"2021-11-25T04:00:04.518743Z","iopub.status.idle":"2021-11-25T04:00:04.578984Z","shell.execute_reply.started":"2021-11-25T04:00:04.518711Z","shell.execute_reply":"2021-11-25T04:00:04.578124Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_csv.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-25T04:00:10.768955Z","iopub.execute_input":"2021-11-25T04:00:10.769223Z","iopub.status.idle":"2021-11-25T04:00:10.784187Z","shell.execute_reply.started":"2021-11-25T04:00:10.769195Z","shell.execute_reply":"2021-11-25T04:00:10.783251Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# total number of bounding boxes\ntotal_num_boxes = train_csv.num_boxes.sum()\ntotal_num_boxes","metadata":{"execution":{"iopub.status.busy":"2021-11-25T04:07:35.375033Z","iopub.execute_input":"2021-11-25T04:07:35.375265Z","iopub.status.idle":"2021-11-25T04:07:35.381930Z","shell.execute_reply.started":"2021-11-25T04:07:35.375238Z","shell.execute_reply":"2021-11-25T04:07:35.380987Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# extract the first sample in each group\nsamples = train_csv.groupby('num_boxes').first()\nsamples","metadata":{"execution":{"iopub.status.busy":"2021-11-25T04:01:04.845990Z","iopub.execute_input":"2021-11-25T04:01:04.846549Z","iopub.status.idle":"2021-11-25T04:01:04.866538Z","shell.execute_reply.started":"2021-11-25T04:01:04.846525Z","shell.execute_reply":"2021-11-25T04:01:04.865895Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(24, 36))\n\nr, c = 7, 3\nfor index, row in samples.iterrows():\n image_path = row['file_path']\n annot_line = row['annotations']\n plt.subplot(r, c, index + 1)\n dimg = draw_boxes(image_path, annot_line)\n plt.imshow(dimg)\n \nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-25T04:01:29.200553Z","iopub.execute_input":"2021-11-25T04:01:29.202087Z","iopub.status.idle":"2021-11-25T04:01:37.786301Z","shell.execute_reply.started":"2021-11-25T04:01:29.202026Z","shell.execute_reply":"2021-11-25T04:01:37.785322Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Box Location Visualization","metadata":{}},{"cell_type":"code","source":"all_boxes_xy = []\nall_boxes_wh = []\n\nfor index, row in tqdm(train_csv.iterrows(), total=len(train_csv)):\n if row['annotations'] != '[]':\n boxes = decode_annotation(row['annotations'])\n \n for box in boxes:\n all_boxes_xy.append([box[0], box[1]])\n all_boxes_wh.append([box[2], box[3]])\n \nall_boxes_xy = np.array(all_boxes_xy)\nall_boxes_wh = np.array(all_boxes_wh)","metadata":{"execution":{"iopub.status.busy":"2021-11-25T04:10:56.343728Z","iopub.execute_input":"2021-11-25T04:10:56.343964Z","iopub.status.idle":"2021-11-25T04:10:57.624683Z","shell.execute_reply.started":"2021-11-25T04:10:56.343939Z","shell.execute_reply":"2021-11-25T04:10:57.623456Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Summary Statistics: Box Center Coordinates and Box Area","metadata":{}},{"cell_type":"code","source":"box_center_df = pd.DataFrame.from_records(all_boxes_xy, columns=['x', 'y'])\n\nbox_shape_df = pd.DataFrame.from_records(all_boxes_wh, columns=['width', 'height'])\nbox_shape_df['area'] = box_shape_df['width'] * box_shape_df['height']","metadata":{"execution":{"iopub.status.busy":"2021-11-25T04:19:29.528593Z","iopub.execute_input":"2021-11-25T04:19:29.529009Z","iopub.status.idle":"2021-11-25T04:19:29.576773Z","shell.execute_reply.started":"2021-11-25T04:19:29.528968Z","shell.execute_reply":"2021-11-25T04:19:29.576270Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Box's (x, y) coordinate summary statistics

","metadata":{}},{"cell_type":"code","source":"box_center_df.describe()","metadata":{"execution":{"iopub.status.busy":"2021-11-25T04:19:30.566492Z","iopub.execute_input":"2021-11-25T04:19:30.566817Z","iopub.status.idle":"2021-11-25T04:19:30.581794Z","shell.execute_reply.started":"2021-11-25T04:19:30.566793Z","shell.execute_reply":"2021-11-25T04:19:30.580438Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Box's (width, height), and area summary statistics

","metadata":{}},{"cell_type":"code","source":"box_shape_df.describe()","metadata":{"execution":{"iopub.status.busy":"2021-11-25T04:19:55.356844Z","iopub.execute_input":"2021-11-25T04:19:55.357176Z","iopub.status.idle":"2021-11-25T04:19:55.373574Z","shell.execute_reply.started":"2021-11-25T04:19:55.357152Z","shell.execute_reply":"2021-11-25T04:19:55.372659Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Distribution of Box Center on Image (all boxes)","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(28, 16))\nplt.scatter(x=all_boxes_xy[:,0], y=all_boxes_xy[:,1], s=0.5, color = 'red')\nplt.title(\"Distribution of Box Center Coordinate on Image\")\nplt.xlabel(\"X value\")\nplt.ylabel(\"Y value\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-25T04:28:24.135848Z","iopub.execute_input":"2021-11-25T04:28:24.136107Z","iopub.status.idle":"2021-11-25T04:28:24.485866Z","shell.execute_reply.started":"2021-11-25T04:28:24.136079Z","shell.execute_reply":"2021-11-25T04:28:24.484672Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Sequence Preview","metadata":{}},{"cell_type":"code","source":"# calculate number of boxes in each sequence\n# we can find that sequence 29424, 37114, 44160 do not contain any object\ntrain_csv.groupby('sequence')['num_boxes'].sum().sort_values(ascending=False).to_frame().T","metadata":{"execution":{"iopub.status.busy":"2021-11-25T04:30:53.191622Z","iopub.execute_input":"2021-11-25T04:30:53.191892Z","iopub.status.idle":"2021-11-25T04:30:53.210838Z","shell.execute_reply.started":"2021-11-25T04:30:53.191864Z","shell.execute_reply":"2021-11-25T04:30:53.209594Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# sequence length\ntrain_csv.groupby('sequence')['image_id'].count().sort_values(ascending=False).to_frame().T","metadata":{"execution":{"iopub.status.busy":"2021-11-25T04:31:23.699219Z","iopub.execute_input":"2021-11-25T04:31:23.699454Z","iopub.status.idle":"2021-11-25T04:31:23.719058Z","shell.execute_reply.started":"2021-11-25T04:31:23.699431Z","shell.execute_reply":"2021-11-25T04:31:23.717969Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# pick one sequence, and convert it to video and adding the annotations to the video\nsample_seq = train_csv[train_csv.sequence == 22643]\nsample_seq","metadata":{"execution":{"iopub.status.busy":"2021-11-25T04:38:49.897404Z","iopub.execute_input":"2021-11-25T04:38:49.897645Z","iopub.status.idle":"2021-11-25T04:38:49.913644Z","shell.execute_reply.started":"2021-11-25T04:38:49.897602Z","shell.execute_reply":"2021-11-25T04:38:49.913044Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def convert_frames_to_video(files, boxes, save_to,fps):\n \n frame_array = []\n \n print(\"loading ...\")\n for filename, annot_line in tqdm(zip(files, boxes), total=len(files)):\n img = cv2.imread(filename)\n height, width, layers = img.shape\n size = (width,height)\n \n boxes = decode_annotation(annot_line)\n \n coords = [] \n for box in boxes: \n coord = [] \n coord.append(box[0]) \n coord.append(box[1]) \n coord.append(box[0] + box[2]) \n coord.append(box[1] + box[3]) \n coords.append(coord) \n\n imgcp = Image.fromarray(img)\n imgcp_draw = ImageDraw.Draw(imgcp)\n\n for coord in coords:\n imgcp_draw.rectangle(coord, fill = None, outline = \"blue\", width=5)\n \n del imgcp_draw\n \n frame_array.append(np.array(imgcp))\n \n out = cv2.VideoWriter(save_to,cv2.VideoWriter_fourcc(*'DIVX'), fps, size)\n \n print(f\"writing to {save_to}\")\n for i in tqdm(range(len(frame_array))):\n # writing to a image array\n out.write(frame_array[i])\n out.release()","metadata":{"execution":{"iopub.status.busy":"2021-11-23T03:12:25.386178Z","iopub.execute_input":"2021-11-23T03:12:25.38648Z","iopub.status.idle":"2021-11-23T03:12:25.397862Z","shell.execute_reply.started":"2021-11-23T03:12:25.386446Z","shell.execute_reply":"2021-11-23T03:12:25.396794Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"at current stage, you might need to download the output sequence...","metadata":{}},{"cell_type":"code","source":"convert_frames_to_video(sample_seq['file_path'].values.tolist(), sample_seq['annotations'].values.tolist(), './sequence.avi', 25)","metadata":{"execution":{"iopub.status.busy":"2021-11-23T03:12:25.99408Z","iopub.execute_input":"2021-11-23T03:12:25.99474Z","iopub.status.idle":"2021-11-23T03:13:01.512386Z","shell.execute_reply.started":"2021-11-23T03:12:25.994689Z","shell.execute_reply":"2021-11-23T03:13:01.511507Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}