{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"
\n \n

Tensorflow - Help Protect the Great Barrier Reef 🌟🐠

\n
\n\n
\n

Introduction

\n

Why this Competition ❓

\n

\n Australia's stunningly beautiful Great Barrier Reef is the world’s largest coral reef and home to 1,500 species of fish, 400 species of corals, 130 species of sharks, rays, and a massive variety of other sea life.Unfortunately, the reef is under threat, in part because of the overpopulation of one particular starfish – the coral-eating crown-of-thorns starfish (or COTS for short).\n

\n

Goal of Competition πŸ₯…

\n

\n The goal of this competition is to accurately identify starfish in real-time by building an object detection model trained on underwater videos of coral reefs.\n

\n
\n\n
\n

Sponsors πŸ’°

\n

TensorFlow

\n

\n TensorFlow is a free and open-source software library for machine learning and artificial intelligence. It can be used across a range of tasks but has a particular focus on training and inference of deep neural networks.\n

\n

CSIRO

\n

\n The Commonwealth Scientific and Industrial Research Organisation is an Australian Government agency responsible for scientific research. CSIRO works with leading organisations around the world.\n

\n

GBRF

\n

\n The Great Barrier Reef Foundation is an Australian non-profit organisation established in 1999 to help protect and preserve the Great Barrier Reef. \n

\n
\n\n

More Details

\n

Check competition page for details

\n

Let's dive in ⬇️

\n\n
","metadata":{}},{"cell_type":"markdown","source":"

Install Libraries ⏬

","metadata":{}},{"cell_type":"code","source":"!pip install --upgrade wandb","metadata":{"_kg_hide-output":true,"scrolled":true,"execution":{"iopub.status.busy":"2021-11-23T11:36:47.892191Z","iopub.execute_input":"2021-11-23T11:36:47.892601Z","iopub.status.idle":"2021-11-23T11:36:58.869213Z","shell.execute_reply.started":"2021-11-23T11:36:47.892498Z","shell.execute_reply":"2021-11-23T11:36:58.867895Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Import Libraries πŸ“š

","metadata":{}},{"cell_type":"code","source":"import os\nfrom kaggle_secrets import UserSecretsClient\nimport wandb\n\nimport numpy as np\nimport ast\n\nimport pandas as pd\nfrom pandas_profiling import ProfileReport\n\nimport matplotlib.pyplot as plt\nfrom matplotlib import rc\nfrom matplotlib.animation import FuncAnimation\nimport matplotlib.animation as animation\nimport seaborn as sns\n\nimport cv2\nfrom IPython.display import Video\n\nfrom tqdm import tqdm\nimport warnings\nwarnings.filterwarnings('ignore')\nsns.set()\n%matplotlib inline","metadata":{"execution":{"iopub.status.busy":"2021-11-23T11:43:04.034652Z","iopub.execute_input":"2021-11-23T11:43:04.03501Z","iopub.status.idle":"2021-11-23T11:43:05.202018Z","shell.execute_reply.started":"2021-11-23T11:43:04.034972Z","shell.execute_reply":"2021-11-23T11:43:05.200855Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Initialize Constants πŸ”°

","metadata":{}},{"cell_type":"code","source":"TRAIN_PATH = \"../input/tensorflow-great-barrier-reef/train_images\"\nHEIGHT, WIDTH = 720, 1280\n\nvid0_path = \"../input/tensorflow-great-barrier-reef/train_images/video_0\"\nvid1_path = \"../input/tensorflow-great-barrier-reef/train_images/video_1\"\nvid2_path = \"../input/tensorflow-great-barrier-reef/train_images/video_2\"\nvid_paths = [vid0_path, vid1_path, vid2_path]\n\nvid0_ls = [os.path.join(vid0_path,f) for f in os.listdir(vid0_path)]\nvid0_ls = sorted(vid0_ls, key=lambda x: int(\"\".join([i for i in x if i.isdigit()])))\n\nvid1_ls = [os.path.join(vid1_path,f) for f in os.listdir(vid1_path)]\nvid1_ls = sorted(vid1_ls, key=lambda x: int(\"\".join([i for i in x if i.isdigit()])))\n\nvid2_ls = [os.path.join(vid2_path,f) for f in os.listdir(vid2_path)]\nvid2_ls = sorted(vid2_ls, key=lambda x: int(\"\".join([i for i in x if i.isdigit()])))\nfiles_ls = [vid0_ls, vid1_ls, vid2_ls]\n\ntrain_df = pd.read_csv(\"../input/tensorflow-great-barrier-reef/train.csv\")\n\nprint(f\"Number of images in First Video : {len(vid0_ls)}\\nNumber of images in Second Video : {len(vid1_ls)}\\nNumber of images in Third Video : {len(vid2_ls)} \\n\\n\")\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-23T11:37:00.84495Z","iopub.execute_input":"2021-11-23T11:37:00.845213Z","iopub.status.idle":"2021-11-23T11:37:01.327867Z","shell.execute_reply.started":"2021-11-23T11:37:00.845183Z","shell.execute_reply":"2021-11-23T11:37:01.326525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Data Preprocessing βš’οΈπŸ”¬

","metadata":{}},{"cell_type":"markdown","source":"

Convert Images to Video

","metadata":{}},{"cell_type":"code","source":"video = cv2.VideoWriter(f\"./video_0.avi\",0,1,(WIDTH, HEIGHT))\nfor f in tqdm(files_ls[0]):\n video.write(cv2.imread(f))","metadata":{"execution":{"iopub.status.busy":"2021-11-23T11:37:01.329784Z","iopub.execute_input":"2021-11-23T11:37:01.330449Z","iopub.status.idle":"2021-11-23T11:40:50.873679Z","shell.execute_reply.started":"2021-11-23T11:37:01.330391Z","shell.execute_reply":"2021-11-23T11:40:50.871588Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Bounding box Metadata

","metadata":{}},{"cell_type":"code","source":"width, height = [], []\nxloc, yloc = [], []\ncount_fish = []\nannot, paths = [], []\n\nfor i in range(3):\n w,h,x,y = [], [], [], []\n c = []\n for f in tqdm(files_ls[i]):\n name = str(i)+\"-\" + str(f.split(\"/\")[-1].split(\".\")[0])\n ls = list(train_df[train_df[\"image_id\"] == name][\"annotations\"])[0]\n ls = ast.literal_eval(ls)\n c.append(len(ls))\n if len(ls) != 0:\n for l in ls:\n w.append(l[\"width\"]); h.append(l[\"height\"]); x.append(l[\"x\"]); y.append(l[\"y\"])\n if i == 0 and len(ls)==1:\n paths.append(f)\n annot.append([l[\"x\"],l[\"y\"],l[\"x\"]+l[\"width\"],l[\"y\"]+l[\"height\"],])\n width.append(w);height.append(h);xloc.append(x);yloc.append(y)\n count_fish.append(c)","metadata":{"execution":{"iopub.status.busy":"2021-11-23T11:40:50.878356Z","iopub.execute_input":"2021-11-23T11:40:50.879823Z","iopub.status.idle":"2021-11-23T11:42:37.459049Z","shell.execute_reply.started":"2021-11-23T11:40:50.879744Z","shell.execute_reply":"2021-11-23T11:42:37.458002Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Data Visualization πŸ“ŠπŸ’Ή

","metadata":{}},{"cell_type":"markdown","source":"
\n

Weights and Biases

\n
\n Checkout Dashboard created for this notebook\n

\n Weights and Biases is a set of Machine Learning tools used for experiment tracking, dataset versioning, and collaborating on ML projects. Weights and Biases is useful in many applications such as\n

\n \n Go to offocial website for more tutorials and Documentation\n
","metadata":{}},{"cell_type":"code","source":"# Check https://www.kaggle.com/debarshichanda/pytorch-w-b-pawpularity-training for more details\ntry:\n user_secrets = UserSecretsClient()\n api_key = user_secrets.get_secret(\"wandb_api\")\n wandb.login(key=api_key)\n anony = None\nexcept:\n anony = \"must\"\n print('''If you want to use your W&B account, Follow these steps :\n -> go to Add-ons {Below name of notebook} -> Secrets -> Add a new Secret\n -> Label = wandb_api\n -> Value = W&B access token from https://wandb.ai/authorize \n ''')","metadata":{"execution":{"iopub.status.busy":"2021-11-23T11:43:13.447915Z","iopub.execute_input":"2021-11-23T11:43:13.448483Z","iopub.status.idle":"2021-11-23T11:43:14.144162Z","shell.execute_reply.started":"2021-11-23T11:43:13.448437Z","shell.execute_reply":"2021-11-23T11:43:14.143291Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"PROJECT_NAME = \"Protect Great Barrier Reef\"\nRAW_LOCATION = \"display_data\"\n\nrun = wandb.init(project=PROJECT_NAME, job_type=\"Sample of 1000 images\", group=\"Data Visualization\")\n\nartifact = wandb.Artifact(RAW_LOCATION, type=\"raw_data\")\nartifact.add_file(\"../input/tensorflow-great-barrier-reef/train.csv\")\n\nwb_table = wandb.Table(columns=[\"Video Id\",\"Image Name\",\"Image\",\"Annotations\"])\n\ntmp_df = train_df.sample(1000, random_state=2021).reset_index(drop=True)\nfor i in range(len(tmp_df)):\n im_id = tmp_df.iloc[i][\"image_id\"]\n loc_fish = tmp_df.iloc[i][\"annotations\"]\n video = int(im_id.split(\"-\")[0])\n count = im_id.split(\"-\")[1]\n impath = os.path.join(vid_paths[video],count+\".jpg\")\n \n wb_table.add_data(video, int(count), wandb.Image(impath), loc_fish)\n \nartifact.add(wb_table, \"Table of 1000 images\")\nrun.log_artifact(artifact)\nrun.finish()","metadata":{"execution":{"iopub.status.busy":"2021-11-23T11:47:59.277352Z","iopub.execute_input":"2021-11-23T11:47:59.278018Z","iopub.status.idle":"2021-11-23T11:49:12.652744Z","shell.execute_reply.started":"2021-11-23T11:47:59.277962Z","shell.execute_reply":"2021-11-23T11:49:12.651978Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from IPython import display\nframe = display.IFrame(run.url, width=1280, height=720)\n''' \nFollow this steps to view Table created\n Artifacts : left side bottom button -> raw_data -> v0 -> Table of 1000 images\n'''\nframe","metadata":{"execution":{"iopub.status.busy":"2021-11-23T11:53:13.495048Z","iopub.execute_input":"2021-11-23T11:53:13.495506Z","iopub.status.idle":"2021-11-23T11:53:13.504498Z","shell.execute_reply.started":"2021-11-23T11:53:13.495464Z","shell.execute_reply":"2021-11-23T11:53:13.503346Z"},"_kg_hide-input":false,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n

Pandas Profiling

\n
\n

\n Pandas profiling is an open source Python module with which we can quickly do an exploratory data analysis with just a few lines of code. Besides, if this is not enough to convince us to use this tool, it also generates interactive reports in web format that can be presented to any person, even if they don’t know programming.\n

\n Go to offocial website for documentation\n
","metadata":{}},{"cell_type":"code","source":"train_report = ProfileReport(train_df,title=\"Metadata of Training images\")\ntrain_report.to_file(\"./train_metadata.html\")\ntrain_report","metadata":{"execution":{"iopub.status.busy":"2021-11-23T11:54:10.882014Z","iopub.execute_input":"2021-11-23T11:54:10.882902Z","iopub.status.idle":"2021-11-23T11:54:26.153727Z","shell.execute_reply.started":"2021-11-23T11:54:10.882842Z","shell.execute_reply":"2021-11-23T11:54:26.152878Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Visualize video frame by frame

","metadata":{}},{"cell_type":"code","source":"rc('animation', html='jshtml') \n\ndef get_animation(paths, annot):\n fig = plt.figure(figsize=(8,8))\n arr = cv2.imread(paths[0],cv2.COLOR_BGR2RGB)\n cv2.rectangle(arr, (annot[0][0],annot[0][1]), (annot[0][2],annot[0][3]),(255,0,0),2)\n im_arr = plt.imshow(arr, cmap=\"gray\")\n plt.axis(\"off\")\n plt.title(\"FLAIR\")\n def update(i):\n arr = cv2.imread(paths[i], cv2.COLOR_BGR2RGB)\n cv2.rectangle(arr, (annot[i][0],annot[i][1]), (annot[i][2],annot[i][3]),(255,0,0),2)\n im_arr.set_array(arr)\n return [im_arr]\n anim = animation.FuncAnimation(fig,update,frames=len(paths),interval=50)\n anim.save(\"./video_0.mp4\")\n return anim\n\nget_animation(paths[::20], annot[::20])","metadata":{"execution":{"iopub.status.busy":"2021-11-23T11:54:47.45992Z","iopub.execute_input":"2021-11-23T11:54:47.460226Z","iopub.status.idle":"2021-11-23T11:55:17.773103Z","shell.execute_reply.started":"2021-11-23T11:54:47.460196Z","shell.execute_reply":"2021-11-23T11:55:17.772004Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Location of Star Fish

","metadata":{}},{"cell_type":"code","source":"fig, axes = plt.subplots(3,2, figsize=(18, 12))\nfig.suptitle(\"Location of Star Fish in 3 videos\")\n\naxes[0,0].hist(xloc[0])\naxes[0,0].set_title(\"X : Video 0\")\naxes[0,1].hist(yloc[0])\naxes[0,1].set_title(\"Y : Video 0\")\n\naxes[1,0].hist(xloc[1])\naxes[1,0].set_title(\"X : Video 1\")\naxes[1,1].hist(yloc[1])\naxes[1,1].set_title(\"Y : Video 1\")\n\naxes[2,0].hist(xloc[2])\naxes[2,0].set_title(\"X : Video 2\")\naxes[2,1].hist(yloc[2])\naxes[2,1].set_title(\"Y : Video 2\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-23T11:55:17.775349Z","iopub.execute_input":"2021-11-23T11:55:17.775958Z","iopub.status.idle":"2021-11-23T11:55:19.517922Z","shell.execute_reply.started":"2021-11-23T11:55:17.775918Z","shell.execute_reply":"2021-11-23T11:55:19.516845Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Size of Star Fish

","metadata":{}},{"cell_type":"code","source":"fig, axes = plt.subplots(3,2, figsize=(18, 12))\nfig.suptitle(\"Size of Star Fish in 3 videos\")\n\naxes[0,0].hist(width[0])\naxes[0,0].set_title(\"Width : Video 0\")\naxes[0,1].hist(height[0])\naxes[0,1].set_title(\"Height : Video 0\")\n\naxes[1,0].hist(width[1])\naxes[1,0].set_title(\"Width : Video 1\")\naxes[1,1].hist(height[1])\naxes[1,1].set_title(\"Height : Video 1\")\n\naxes[2,0].hist(width[2])\naxes[2,0].set_title(\"Width : Video 2\")\naxes[2,1].hist(height[2])\naxes[2,1].set_title(\"Height : Video 2\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-23T11:55:19.519244Z","iopub.execute_input":"2021-11-23T11:55:19.519501Z","iopub.status.idle":"2021-11-23T11:55:21.142359Z","shell.execute_reply.started":"2021-11-23T11:55:19.519468Z","shell.execute_reply":"2021-11-23T11:55:21.14146Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Count of Star Fish

","metadata":{}},{"cell_type":"code","source":"fig, axes = plt.subplots(1,3, figsize=(18, 6))\nfig.suptitle(\"Count of Star Fish in 3 videos\")\n\naxes[0].hist(count_fish[0])\naxes[0].set_title(\"Video 0\")\naxes[1].hist(count_fish[1])\naxes[1].set_title(\"Video 1\")\naxes[2].hist(count_fish[2])\naxes[2].set_title(\"Video 2\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-23T11:55:21.144422Z","iopub.execute_input":"2021-11-23T11:55:21.145007Z","iopub.status.idle":"2021-11-23T11:55:21.636034Z","shell.execute_reply.started":"2021-11-23T11:55:21.144959Z","shell.execute_reply":"2021-11-23T11:55:21.634879Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Sample Images with Bounding Boxes

","metadata":{}},{"cell_type":"code","source":"fig, axes = plt.subplots(3,3, figsize=(18,12))\nimage_ids = [\"0-16\",\"0-17\",\"0-18\",\"0-19\",\"0-20\",\"0-21\",\"0-22\",\"0-23\",\"0-24\"]\n\nfor i in range(9):\n r,c = i//3, i%3\n name = image_ids[i]\n im_path = os.path.join(vid0_path,name.split(\"-\")[1]+\".jpg\")\n ls = list(train_df[train_df[\"image_id\"]==name][\"annotations\"])[0]\n ls = ast.literal_eval(ls)\n img = cv2.imread(im_path, cv2.COLOR_BGR2RGB)\n for box in ls:\n x1 = box[\"x\"]\n y1 = box[\"y\"]\n x2 = x1 + box[\"width\"]\n y2 = y1 + box[\"height\"]\n cv2.rectangle(img, (x1,y1), (x2,y2),(255,0,0),2)\n axes[r,c].imshow(img)\n axes[r,c].axis(\"off\")\n axes[r,c].set_title(f\"Image : {image_ids[i]}\",fontsize=15, weight='bold')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-23T11:55:21.637584Z","iopub.execute_input":"2021-11-23T11:55:21.638068Z","iopub.status.idle":"2021-11-23T11:55:24.59106Z","shell.execute_reply.started":"2021-11-23T11:55:21.638016Z","shell.execute_reply":"2021-11-23T11:55:24.590072Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n

If you come this far, you could've got some insights from this notebook. An upvote would be very helpful :). Kindly comment if there are any doubts or mistakes

\n\n
","metadata":{}}]}