{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# [Tensorflow - Help Protect the Great Barrier Reef](https://www.kaggle.com/c/tensorflow-great-barrier-reef)\n\n<img src=\"https://storage.googleapis.com/kaggle-competitions/kaggle/31703/logos/header.png?t=2021-10-29-00-30-04\">","metadata":{}},{"cell_type":"markdown","source":"# GOAL","metadata":{}},{"cell_type":"markdown","source":"### 1. Accurately identify starfish in real-time by building an object detection model trained on underwater videos of coral reefs.\n\n### 2. Predict the presence and position  crown-of-thorns starfish in sequences of underwater images taken at various times and locations around the Great Barrier Reef.","metadata":{}},{"cell_type":"markdown","source":"# NEED","metadata":{}},{"cell_type":"markdown","source":"### 1. Great Barrier Reef is under threat because of the overpopulation of one particular starfish – the coral-eating crown-of-thorns starfish (or COTS for short). \n\n### 2. Underwater cameras will collect thousands of reef images and AI technology could drastically improve the efficiency and scale at which reef managers detect and control COTS outbreaks.","metadata":{}},{"cell_type":"markdown","source":"# INSTRUCTIONS","metadata":{}},{"cell_type":"markdown","source":"### 1. Predictions take the form of a bounding box together with a confidence score for each identified starfish. \n\n### 2. An image may contain zero or more starfish.\n\n### 3. This competition uses a hidden test set that will be served by an API to ensure you evaluate the images in the same order they were recorded within each video. When your submitted notebook is scored, the actual test data (including a sample submission) will be availabe to your notebook.","metadata":{}},{"cell_type":"markdown","source":"# IMPORTING LIBRARIES","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns","metadata":{"execution":{"iopub.status.busy":"2022-01-13T15:50:04.688572Z","iopub.execute_input":"2022-01-13T15:50:04.688917Z","iopub.status.idle":"2022-01-13T15:50:05.522995Z","shell.execute_reply.started":"2022-01-13T15:50:04.688886Z","shell.execute_reply":"2022-01-13T15:50:05.522280Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from PIL import Image, ImageDraw\nimport random","metadata":{"execution":{"iopub.status.busy":"2022-01-13T16:18:27.260619Z","iopub.execute_input":"2022-01-13T16:18:27.261186Z","iopub.status.idle":"2022-01-13T16:18:27.264944Z","shell.execute_reply.started":"2022-01-13T16:18:27.261147Z","shell.execute_reply":"2022-01-13T16:18:27.263791Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# DATA DESCRIPTION","metadata":{}},{"cell_type":"markdown","source":"* `video_id` - ID number of the video the image was part of. The video ids are not meaningfully ordered.\n* `video_frame` - The frame number of the image within the video. Expect to see occasional gaps in the frame number from when the diver surfaced.\n* `sequence` - ID of a gap-free subset of a given video. The sequence ids are not meaningfully ordered.\n* `sequence_frame` - The frame number within a given sequence.\n* `image_id` - ID code for the image, in the format `{video_id}-{video_frame}`\n* `annotations` - The bounding boxes of any starfish detections in a string format that can be evaluated directly with Python. Does not use the same format as the predictions you will submit. Not available in test.csv. A bounding box is described by the pixel coordinate `(x_min, y_min)` of its lower left corner within the image together with its `width` and `height` in pixels --> (COCO format).","metadata":{}},{"cell_type":"code","source":"train_metaData = pd.read_csv('../input/tensorflow-great-barrier-reef/train.csv')\ntrain_metaData.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T15:50:05.524435Z","iopub.execute_input":"2022-01-13T15:50:05.524681Z","iopub.status.idle":"2022-01-13T15:50:05.608287Z","shell.execute_reply.started":"2022-01-13T15:50:05.524645Z","shell.execute_reply":"2022-01-13T15:50:05.607592Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_metaData = pd.read_csv('../input/tensorflow-great-barrier-reef/test.csv')\ntest_metaData.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T15:50:05.609675Z","iopub.execute_input":"2022-01-13T15:50:05.609926Z","iopub.status.idle":"2022-01-13T15:50:05.624545Z","shell.execute_reply.started":"2022-01-13T15:50:05.609893Z","shell.execute_reply":"2022-01-13T15:50:05.623869Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# DATA VISUALIZATION","metadata":{}},{"cell_type":"code","source":"df_train = train_metaData.copy()\ntrain_dir = \"../input/tensorflow-great-barrier-reef/train_images\"\ndf_train['image_path'] = train_dir + \"/video_\" + df_train['video_id'].astype(str) + \"/\" + df_train['video_frame'].astype(str) + \".jpg\"\ndf_train.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T15:50:07.215325Z","iopub.execute_input":"2022-01-13T15:50:07.218013Z","iopub.status.idle":"2022-01-13T15:50:07.356066Z","shell.execute_reply.started":"2022-01-13T15:50:07.217970Z","shell.execute_reply":"2022-01-13T15:50:07.355248Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train['video_id'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T15:50:07.360195Z","iopub.execute_input":"2022-01-13T15:50:07.362137Z","iopub.status.idle":"2022-01-13T15:50:07.377156Z","shell.execute_reply.started":"2022-01-13T15:50:07.362090Z","shell.execute_reply":"2022-01-13T15:50:07.376198Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.bar(x = df_train['video_id'].value_counts().index, height = df_train['video_id'].value_counts())","metadata":{"execution":{"iopub.status.busy":"2022-01-13T15:50:07.381330Z","iopub.execute_input":"2022-01-13T15:50:07.383213Z","iopub.status.idle":"2022-01-13T15:50:07.636553Z","shell.execute_reply.started":"2022-01-13T15:50:07.383169Z","shell.execute_reply":"2022-01-13T15:50:07.635893Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.info()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T15:50:07.639061Z","iopub.execute_input":"2022-01-13T15:50:07.639651Z","iopub.status.idle":"2022-01-13T15:50:07.663537Z","shell.execute_reply.started":"2022-01-13T15:50:07.639610Z","shell.execute_reply":"2022-01-13T15:50:07.662766Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Number of training images","metadata":{}},{"cell_type":"code","source":"num_training_images = len(df_train)\nnum_training_images","metadata":{"execution":{"iopub.status.busy":"2022-01-13T15:50:07.664635Z","iopub.execute_input":"2022-01-13T15:50:07.664878Z","iopub.status.idle":"2022-01-13T15:50:07.672050Z","shell.execute_reply.started":"2022-01-13T15:50:07.664845Z","shell.execute_reply":"2022-01-13T15:50:07.671227Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Looking at random images from every video.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize = (20, 20))\nfor i in range(0, 10):\n    plt.subplot(5, 2, i+1)\n    index = random.randint(0, 23501)\n    img_path = df_train['image_path'].iloc[index]\n    img = Image.open(img_path)\n    plt.imshow(img)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T17:14:06.851689Z","iopub.execute_input":"2022-01-13T17:14:06.852241Z","iopub.status.idle":"2022-01-13T17:14:10.430117Z","shell.execute_reply.started":"2022-01-13T17:14:06.852202Z","shell.execute_reply":"2022-01-13T17:14:10.428232Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_video_2_10 = plt.imread('../input/tensorflow-great-barrier-reef/train_images/video_2/10.jpg')\nimg_video_2_10.shape","metadata":{"execution":{"iopub.status.busy":"2022-01-13T17:12:30.095980Z","iopub.execute_input":"2022-01-13T17:12:30.096501Z","iopub.status.idle":"2022-01-13T17:12:30.131670Z","shell.execute_reply.started":"2022-01-13T17:12:30.096464Z","shell.execute_reply":"2022-01-13T17:12:30.130986Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"All images are of same size so need of resizing.  \nAs you see, Images are really blur and definitely needs to be enhanced before modelling.","metadata":{}},{"cell_type":"code","source":"df_train_annotated = df_train[df_train['annotations'] != '[]']\ndf_train_annotated","metadata":{"execution":{"iopub.status.busy":"2022-01-13T15:50:07.673938Z","iopub.execute_input":"2022-01-13T15:50:07.674590Z","iopub.status.idle":"2022-01-13T15:50:07.699026Z","shell.execute_reply.started":"2022-01-13T15:50:07.674550Z","shell.execute_reply":"2022-01-13T15:50:07.698301Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"(len(df_train_annotated)/ num_training_images) * 100","metadata":{"execution":{"iopub.status.busy":"2022-01-13T15:50:07.700683Z","iopub.execute_input":"2022-01-13T15:50:07.701134Z","iopub.status.idle":"2022-01-13T15:50:07.706954Z","shell.execute_reply.started":"2022-01-13T15:50:07.701098Z","shell.execute_reply":"2022-01-13T15:50:07.706049Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Only about 20% of the data is annotated.","metadata":{}},{"cell_type":"markdown","source":"Annotations contains the co-ordinate values for the bounding boxes where we will find starfishes.","metadata":{}},{"cell_type":"code","source":"df_train.dtypes","metadata":{"execution":{"iopub.status.busy":"2022-01-13T15:50:07.708680Z","iopub.execute_input":"2022-01-13T15:50:07.709205Z","iopub.status.idle":"2022-01-13T15:50:07.718686Z","shell.execute_reply.started":"2022-01-13T15:50:07.709165Z","shell.execute_reply":"2022-01-13T15:50:07.717743Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now, we will look at the number of bounding boxes in each image because it is clearly stated that an image may conatin zero or more starfish.","metadata":{}},{"cell_type":"code","source":"df_train['No_bbox'] = df_train['annotations'].apply(lambda x:x.count('{')) \ndf_train.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T15:50:07.720367Z","iopub.execute_input":"2022-01-13T15:50:07.720731Z","iopub.status.idle":"2022-01-13T15:50:07.750426Z","shell.execute_reply.started":"2022-01-13T15:50:07.720693Z","shell.execute_reply":"2022-01-13T15:50:07.749714Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train['No_bbox'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T15:50:07.751700Z","iopub.execute_input":"2022-01-13T15:50:07.751976Z","iopub.status.idle":"2022-01-13T15:50:07.759494Z","shell.execute_reply.started":"2022-01-13T15:50:07.751944Z","shell.execute_reply":"2022-01-13T15:50:07.758544Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (10,5))\nsns.countplot(x = df_train['No_bbox'])","metadata":{"execution":{"iopub.status.busy":"2022-01-13T15:50:07.761073Z","iopub.execute_input":"2022-01-13T15:50:07.761715Z","iopub.status.idle":"2022-01-13T15:50:08.046308Z","shell.execute_reply.started":"2022-01-13T15:50:07.761617Z","shell.execute_reply":"2022-01-13T15:50:08.045607Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Doing the same for the annotated data","metadata":{}},{"cell_type":"code","source":"df_train_annotated['No_bbox'] = df_train_annotated['annotations'].apply(lambda x:x.count('{')) \ndf_train_annotated.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T15:50:08.047632Z","iopub.execute_input":"2022-01-13T15:50:08.047894Z","iopub.status.idle":"2022-01-13T15:50:08.065296Z","shell.execute_reply.started":"2022-01-13T15:50:08.047859Z","shell.execute_reply":"2022-01-13T15:50:08.064563Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Doing this to get **indexes** of those images which contain greater number of bounding boxes for **visualization**.","metadata":{}},{"cell_type":"code","source":"df_train_annotated[df_train_annotated['No_bbox'] >= 5]","metadata":{"execution":{"iopub.status.busy":"2022-01-13T16:16:18.980985Z","iopub.execute_input":"2022-01-13T16:16:18.981661Z","iopub.status.idle":"2022-01-13T16:16:19.001137Z","shell.execute_reply.started":"2022-01-13T16:16:18.981625Z","shell.execute_reply":"2022-01-13T16:16:19.000493Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train_annotated[df_train_annotated['No_bbox'] >= 8]","metadata":{"execution":{"iopub.status.busy":"2022-01-13T17:38:49.558684Z","iopub.execute_input":"2022-01-13T17:38:49.559518Z","iopub.status.idle":"2022-01-13T17:38:49.576017Z","shell.execute_reply.started":"2022-01-13T17:38:49.559476Z","shell.execute_reply":"2022-01-13T17:38:49.575213Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# this module helps to find out programmatically what the current grammar looks like.\nimport ast\nast.literal_eval(df_train_annotated.iloc[2345].annotations)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T17:27:20.767725Z","iopub.execute_input":"2022-01-13T17:27:20.768208Z","iopub.status.idle":"2022-01-13T17:27:20.774280Z","shell.execute_reply.started":"2022-01-13T17:27:20.768171Z","shell.execute_reply":"2022-01-13T17:27:20.773600Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To know more about **ast** library see:\n1. https://docs.python.org/3/library/ast.html\n2. https://stackoverflow.com/questions/29552950/when-to-use-ast-literal-eval/29556591","metadata":{}},{"cell_type":"code","source":"def visualize_img_annots(df, id):\n    img_path = df['image_path'][id]\n    img = Image.open(img_path)\n    bounding_boxes = ast.literal_eval(df['annotations'].loc[id])\n                                         \n    for box in bounding_boxes:\n            shape = (box['x'], box['y'], box['x']+box['width'], box['y']+box['height'])\n            ImageDraw.Draw(img).rectangle(shape, outline=180, width=3)\n    display(img)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T17:35:53.039968Z","iopub.execute_input":"2022-01-13T17:35:53.040658Z","iopub.status.idle":"2022-01-13T17:35:53.046096Z","shell.execute_reply.started":"2022-01-13T17:35:53.040623Z","shell.execute_reply":"2022-01-13T17:35:53.045333Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Number of bounding boxes = 5\nvisualize_img_annots(df_train_annotated, id=19824)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T17:39:29.398415Z","iopub.execute_input":"2022-01-13T17:39:29.398685Z","iopub.status.idle":"2022-01-13T17:39:29.761742Z","shell.execute_reply.started":"2022-01-13T17:39:29.398651Z","shell.execute_reply":"2022-01-13T17:39:29.761141Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Number of bounding boxes = 11\nvisualize_img_annots(df_train_annotated, id = 9292)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T17:40:14.638692Z","iopub.execute_input":"2022-01-13T17:40:14.639253Z","iopub.status.idle":"2022-01-13T17:40:14.976591Z","shell.execute_reply.started":"2022-01-13T17:40:14.639210Z","shell.execute_reply":"2022-01-13T17:40:14.975986Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# NOTEBOOK STILL IN PROGRESS","metadata":{}}]}