{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"
\n \n

Tensorflow - Help Protect the Great Barrier Reef ๐ŸŒŸ๐Ÿ 

\n
\n\n
\n

Introduction

\n

Why this Competition โ“

\n

\n Australia's stunningly beautiful Great Barrier Reef is the worldโ€™s largest coral reef and home to 1,500 species of fish, 400 species of corals, 130 species of sharks, rays, and a massive variety of other sea life.Unfortunately, the reef is under threat, in part because of the overpopulation of one particular starfish โ€“ the coral-eating crown-of-thorns starfish (or COTS for short).\n

\n

Goal of Competition ๐Ÿฅ…

\n

\n The goal of this competition is to accurately identify starfish in real-time by building an object detection model trained on underwater videos of coral reefs.\n

\n
\n\n
\n

Sponsors ๐Ÿ’ฐ

\n

TensorFlow

\n

\n TensorFlow is a free and open-source software library for machine learning and artificial intelligence. It can be used across a range of tasks but has a particular focus on training and inference of deep neural networks.\n

\n

CSIRO

\n

\n The Commonwealth Scientific and Industrial Research Organisation is an Australian Government agency responsible for scientific research. CSIRO works with leading organisations around the world.\n

\n

GBRF

\n

\n The Great Barrier Reef Foundation is an Australian non-profit organisation established in 1999 to help protect and preserve the Great Barrier Reef. \n

\n
\n\n

More Details

\n

Check competition page for details

\n

Let's dive in โฌ‡๏ธ

\n\n
","metadata":{}},{"cell_type":"markdown","source":"

Install Libraries โฌ

","metadata":{}},{"cell_type":"code","source":"import torch\ncuda_version_major = int(torch.version.cuda.split('.')[0])\n\n!wget https://raw.githubusercontent.com/airctic/icevision/master/icevision_install.sh\n!bash icevision_install.sh cuda11\n!pip install torchtext==0.11.0 --upgrade\n\nimport IPython\nIPython.Application.instance().kernel.do_shutdown(True)","metadata":{"_kg_hide-output":true,"scrolled":true,"execution":{"iopub.status.busy":"2021-12-16T12:40:55.390749Z","iopub.execute_input":"2021-12-16T12:40:55.391075Z","iopub.status.idle":"2021-12-16T12:45:31.336809Z","shell.execute_reply.started":"2021-12-16T12:40:55.390992Z","shell.execute_reply":"2021-12-16T12:45:31.335989Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Import Libraries ๐Ÿ“š

","metadata":{}},{"cell_type":"code","source":"import os\nimport shutil\n\nfrom kaggle_secrets import UserSecretsClient\nimport copy \nimport time\n\nimport pandas as pd\nfrom pandas_profiling import ProfileReport\n\nimport numpy as np\nimport ast\n\nimport cv2\n\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport seaborn as sns\nsns.set()\n\nimport torch\nfrom icevision.all import *\n\nimport wandb\n\nfrom colorama import Fore, Back, Style\n\nfrom IPython.display import IFrame\nfrom tqdm import tqdm\nimport warnings\nwarnings.filterwarnings('ignore')","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:46:33.655535Z","iopub.execute_input":"2021-12-16T12:46:33.655796Z","iopub.status.idle":"2021-12-16T12:46:33.669179Z","shell.execute_reply.started":"2021-12-16T12:46:33.655758Z","shell.execute_reply":"2021-12-16T12:46:33.668485Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Initialize Constants ๐Ÿ”ฐ

","metadata":{}},{"cell_type":"code","source":"TRAIN_PATH = \"../input/tensorflow-great-barrier-reef/train_images\"\nHEIGHT, WIDTH = 720, 1280\nimage_size = 640","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:46:43.910762Z","iopub.execute_input":"2021-12-16T12:46:43.911101Z","iopub.status.idle":"2021-12-16T12:46:43.915505Z","shell.execute_reply.started":"2021-12-16T12:46:43.911061Z","shell.execute_reply":"2021-12-16T12:46:43.91466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_files(path):\n vid_ls = [os.path.join(path,f) for f in os.listdir(path)]\n return sorted(vid_ls, key=lambda x: int(\"\".join([i for i in x if i.isdigit()])))","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:46:44.04325Z","iopub.execute_input":"2021-12-16T12:46:44.043506Z","iopub.status.idle":"2021-12-16T12:46:44.048513Z","shell.execute_reply.started":"2021-12-16T12:46:44.043477Z","shell.execute_reply":"2021-12-16T12:46:44.047692Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"vid_paths = [\n f'{TRAIN_PATH}/video_0',\n f'{TRAIN_PATH}/video_1',\n f'{TRAIN_PATH}/video_2',\n]\n\nfiles_ls = [get_files(vid_path) for vid_path in vid_paths]\nlen(files_ls)\n\ntrain_df = pd.read_csv(\"../input/tensorflow-great-barrier-reef/train.csv\")","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:46:44.192042Z","iopub.execute_input":"2021-12-16T12:46:44.192336Z","iopub.status.idle":"2021-12-16T12:46:44.775114Z","shell.execute_reply.started":"2021-12-16T12:46:44.192306Z","shell.execute_reply":"2021-12-16T12:46:44.774279Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:46:44.778761Z","iopub.execute_input":"2021-12-16T12:46:44.779002Z","iopub.status.idle":"2021-12-16T12:46:44.80357Z","shell.execute_reply.started":"2021-12-16T12:46:44.778971Z","shell.execute_reply":"2021-12-16T12:46:44.802897Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Data Preprocessing โš’๏ธ๐Ÿ”ฌ

","metadata":{}},{"cell_type":"code","source":"def get_oldpath(x):\n return os.path.join(vid_paths[x.video_id], f'{str(x.video_frame)}.jpg')\n\ndef get_newpath(x):\n filename = f\"{x.video_id}_{x.video_frame}.jpg\"\n return os.path.join(\"./dataset\", filename)\n\ndef get_filename(x):\n return f\"{x.video_id}_{x.video_frame}.jpg\"","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:46:44.868967Z","iopub.execute_input":"2021-12-16T12:46:44.869183Z","iopub.status.idle":"2021-12-16T12:46:44.874293Z","shell.execute_reply.started":"2021-12-16T12:46:44.869157Z","shell.execute_reply":"2021-12-16T12:46:44.873399Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# annotation ์ฆ‰ ๋ ˆ์ด๋ธ”์ด ์žˆ๋Š” ๊ฒƒ๋“ค๋งŒ์œผ๋กœ ์ถ”๋ฆฐ๋‹ค (ํ•™์Šต์šฉ ์ด๋‹ˆ๊นŒ)\ntrain_df = train_df[train_df.annotations != \"[]\"]\n\n# annotation ์ •๋ณด๋Š” ๋ฌธ์ž์—ด์ด๋‹ค (์˜ˆ์‹œ.\"[{'x': 559, 'y': 213, 'width': 50, 'height': 32}]\")\n# ์ด๋ฅผ ํŒŒ์ด์ฌ ๊ฐ์ฒด๋กœ ๋ณ€ํ™˜ํ•˜๋Š” ๋ฐฉ๋ฒ• ์ค‘ ํ•˜๋‚˜๋กœ ast.literal_eval ์„ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋‹ค.\ntrain_df[\"annotations\"] = train_df[\"annotations\"].map(lambda x : ast.literal_eval(x))\n\n# DataFrame์˜ apply ๋Š” ๋ ˆ์ฝ”๋“œ๋ณ„๋กœ ์ง€์ •๋œ ํ•จ์ˆ˜๋ฅผ ์ ์šฉํ•œ ๋’ค, ์ „์ฒด๊ฐ€ ์ ์šฉ๋œ ์ƒˆ๋กœ์šด DataFrame์„ ๋ฐ˜ํ™˜ํ•œ๋‹ค.\n# filepath๋Š” ์‹ค์ œ ํŒŒ์ผ์ด ์ €์žฅ๋œ ์œ„์น˜ (์›๋ณธ)\n# newpath๋Š” ์›๋ณธ ํŒŒ์ผ์„ ๋ณต์‚ฌํ•ด ์˜ฎ๊ฒจ๋†“์„ ์œ„์น˜ (๋ณต์‚ฌ๋ณธ)\n# filename์€ ๋‹จ์ˆœํžˆ ํŒŒ์ผ์ด๋ฆ„\ntrain_df[\"filepath\"] = train_df.apply(lambda x : get_oldpath(x), axis=1)\ntrain_df[\"newpath\"] = train_df.apply(lambda x : get_newpath(x), axis=1)\ntrain_df[\"filename\"] = train_df.apply(lambda x : get_filename(x), axis=1)\n\nos.makedirs(\"./dataset\",exist_ok=True)\n\nfor i in tqdm(range(len(train_df))):\n src = train_df.iloc[i][\"filepath\"]\n dst = train_df.iloc[i][\"newpath\"]\n shutil.copy(src,dst)\n \ntrain_df.head(2)","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:46:45.067234Z","iopub.execute_input":"2021-12-16T12:46:45.0675Z","iopub.status.idle":"2021-12-16T12:47:48.306649Z","shell.execute_reply.started":"2021-12-16T12:46:45.067468Z","shell.execute_reply":"2021-12-16T12:47:48.305848Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.iloc[0:2].explode('annotations').apply(lambda x : x.annotations[\"x\"], axis=1)","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:47:48.308279Z","iopub.execute_input":"2021-12-16T12:47:48.308574Z","iopub.status.idle":"2021-12-16T12:47:48.327139Z","shell.execute_reply.started":"2021-12-16T12:47:48.308533Z","shell.execute_reply":"2021-12-16T12:47:48.326247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = train_df\n\n# DataFrame์˜ explode ๋ฉ”์„œ๋“œ๋Š” ๋ฆฌ์ŠคํŠธ๋ฅผ ํ’€์–ดํ—ค์นœ๋‹ค\n# [{'x': 559, 'y': 213, 'width': 50, 'height': 32}] => {'x': 559, 'y': 213, 'width': 50, 'height': 32}\ndf = df.explode(\"annotations\")\n\n# ๋ชจ๋“  ์ด๋ฏธ์ง€์˜ ํฌ๊ธฐ(๋†’์ด ๋„ˆ๋น„), ๋ ˆ์ด๋ธ”์ด ๋ชจ๋‘ ๋™์ผํ•˜๋‹ค\n# ์—‘์…€์—์„œ ๋“œ๋ž˜๊ทธ๋กœ ์ฃผ๋ฃจ๋ฃฉ ๊ฐ™์€ ๊ฐš์„ ์ฑ„์›Œ๋„ฃ๋Š”๊ฒƒ ๊ฐ™์€ ์ผ\ndf[\"width\"] = [WIDTH]*len(df)\ndf[\"height\"] = [HEIGHT]*len(df)\ndf[\"label\"] = [\"starfish\"]*len(df)\n\n# annotation์— ๋‹ด๊ธด x, y, width, height ๋Œ€์‹ , \n# ๊ฐ์ฒดํƒ์ง€์—์„œ ์ผ๋ฐ˜์ ์œผ๋กœ ์‚ฌ์šฉ๋˜๋Š” xmin, ymin, xmax, ymax ์šฉ์–ด์™€ ์•Œ๋งž์€ ๊ฐ’์œผ๋กœ ๋Œ€์ฒดํ•œ๋‹ค\ndf[\"xmin\"] = df.apply(lambda x : x.annotations[\"x\"], axis=1)\ndf[\"ymin\"] = df.apply(lambda x : x.annotations[\"y\"], axis=1)\ndf[\"xmax\"] = df.apply(lambda x : x.annotations[\"x\"]+x.annotations[\"width\"], axis=1)\ndf[\"ymax\"] = df.apply(lambda x : x.annotations[\"y\"]+x.annotations[\"height\"], axis=1)\n\n# ๋ฐ”์šด๋”ฉ๋ฐ•์Šค์˜ ์šฐ์ธก ํ•˜๋‹จ ์ขŒํ‘œ๊ฐ€ \n# ์ด๋ฏธ์ง€์˜ ์šฐ์ธก ํ•˜๋‹จ ๋ชจ์„œ๋ฆฌ๋ฅผ ๋ฒ—์–ด๋‚˜๋Š” ๊ฒฝ์šฐ\ndf.loc[df[\"xmax\"] > 1280, \"xmax\"] = 1280\ndf.loc[df[\"ymax\"] > 720, \"ymax\"] = 720\n\n# ๊ฐ์ฒด ํƒ์ง€์— ๋ถˆํ•„์š”ํ•œ ์—ด์€ ์ œ๊ฑฐํ•œ๋‹ค\ndf = df.drop([\"video_id\",\"sequence\",\"video_frame\",\"sequence_frame\",\"image_id\",\"annotations\",\"filepath\",\"newpath\"], axis=1)\ndf = df.reset_index(drop=True)\ndf.head(3)","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:47:48.32937Z","iopub.execute_input":"2021-12-16T12:47:48.330044Z","iopub.status.idle":"2021-12-16T12:47:49.327624Z","shell.execute_reply.started":"2021-12-16T12:47:48.329996Z","shell.execute_reply":"2021-12-16T12:47:49.326826Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Data Visualization ๐Ÿ“Š๐Ÿ’น

","metadata":{}},{"cell_type":"markdown","source":"
\n

Weights and Biases

\n
\n Checkout Dashboard created for this notebook\n

\n Weights and Biases is a set of Machine Learning tools used for experiment tracking, dataset versioning, and collaborating on ML projects. Weights and Biases is useful in many applications such as\n

\n \n Go to offocial website for more tutorials and Documentation\n
","metadata":{}},{"cell_type":"code","source":"# Check https://www.kaggle.com/debarshichanda/pytorch-w-b-pawpularity-training for more details\n\n# ์ด ์„น์…˜์€ ํ•™์Šต์šฉ ๋ฐ์ดํ„ฐ์…‹์„ W&B ๋Œ€์‹œ๋ณด๋“œ์—์„œ ์‹œ๊ฐํ™”ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ๋ณด์—ฌ์ค€๋‹ค.\n# \n# 1. W&B์˜ API ํ‚ค๋ฅผ ๋ฐœ๊ธ‰ ๋ฐ›๋Š”๋‹ค\n# 2. Add-ons => Secrets ๋ฉ”๋‰ด๋ฅผ ํ†ตํ•ด, wandb_api ๋ผ๋Š” ํ‚ค์— W&B API๊ฐ’์„ ํ• ๋‹นํ•œ๋‹ค\n# 3. ์•„๋ž˜ ์ฝ”๋“œ๋ฅผ ์‹คํ–‰ํ•œ๋‹ค.\n\ntry:\n user_secrets = UserSecretsClient()\n api_key = user_secrets.get_secret(\"wandb_api\")\n wandb.login(key=api_key)\n anony = None\nexcept:\n anony = \"must\"\n print('''If you want to use your W&B account, Follow these steps :\n -> go to Add-ons {Below name of notebook} -> Secrets -> Add a new Secret\n -> Label = wandb_api\n -> Value = W&B access token from https://wandb.ai/authorize \n ''')","metadata":{"execution":{"iopub.status.busy":"2021-12-16T01:50:52.636565Z","iopub.execute_input":"2021-12-16T01:50:52.636931Z","iopub.status.idle":"2021-12-16T01:50:54.540847Z","shell.execute_reply.started":"2021-12-16T01:50:52.636887Z","shell.execute_reply":"2021-12-16T01:50:54.539759Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"WANDB_CONFIG = {\n \"project_name\" : \"Protect Great Barrier Reef\",\n \"group_name\" : \"IceVision Training\",\n \"job_type_data\" : \"Data Visualization\",\n \"job_type_train\" : \"Training\",\n \"anonymity\" : \"must\",\n \"artifact\" : \"training_data\"\n}\n\nrun = wandb.init(\n project = WANDB_CONFIG[\"project_name\"],\n group = WANDB_CONFIG[\"group_name\"],\n job_type = WANDB_CONFIG[\"job_type_data\"],\n anonymous= WANDB_CONFIG[\"anonymity\"]\n)","metadata":{"execution":{"iopub.status.busy":"2021-12-16T01:50:56.440616Z","iopub.execute_input":"2021-12-16T01:50:56.440951Z","iopub.status.idle":"2021-12-16T01:51:04.157901Z","shell.execute_reply.started":"2021-12-16T01:50:56.440917Z","shell.execute_reply":"2021-12-16T01:51:04.156558Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# ๋””์Šคํ”Œ๋ ˆ์ด๋  ํ…Œ์ด๋ธ”์˜ ์—ด์„ ์ •์˜ํ•œ๋‹ค\nwb_table = wandb.Table(columns = [\n \"filename\", \"Image\", \"xmin\", \"ymin\", \"xmax\", \"ymax\"\n])\n\n# ํ•™์Šต์šฉ ๋ฐ์ดํ„ฐ ์ค‘ ์ฒ˜์Œ 100๊ฐœ์— ๋Œ€ํ•ด์„œ tqdm ํ•œ๋‹ค\nfor i in tqdm(range(len(df))[:100]):\n row = df.loc[i]\n impath = os.path.join(\"./dataset\",row[\"filename\"])\n im = PIL.Image.open(impath)\n bbox = [{\n \"position\": {\n \"minX\": int(row['xmin']),\n \"maxX\": int(row[\"xmax\"]),\n \"minY\": int(row[\"ymin\"]),\n \"maxY\": int(row[\"ymax\"])\n },\n \"class_id\": 1,\n \"box_caption\": \"starfish\",\n \"domain\": \"pixel\"\n }]\n \n # ์ด๋ฏธ์ง€๋Š” wandb.Image ๊ฐ์ฒด ํ˜•ํƒœ๊ฐ€ ๋˜์–ด์•ผํ•œ๋‹ค.\n # ๋ณด๋‹ค์‹œํ”ผ PIL์˜ Image ํ˜•์‹์„ ๊ทธ๋Œ€๋กœ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์œผ๋ฉฐ,\n # boxes ๋ฐ classes ๋ผ๋Š” ์ถ”๊ฐ€ ํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ ์„ค์ •ํ•˜๋ฉด ์ด๋ฏธ์ง€๋‚ด ๊ฐ์ฒด ์œ„์น˜ ๋ฐ ๋ฒ”์ฃผ๋ฅผ ๊ฒฐ์ •ํ•ด ๋ณด์—ฌ์ค„ ์ˆ˜ ์žˆ๋‹ค.\n image = wandb.Image(im,\n boxes = {\n \"ground_truth\": {\n \"box_data\": bbox,\n \"class_labels\" : {1: 'starfish'}\n }\n },\n classes = [{\"id\": 0, \"name\": \"background\"}, {\"id\": 1, \"name\": \"starfish\"}]\n )\n \n # ์ž…๋ ฅ๋œ ๋ฐ์ดํ„ฐ๊ฐ€ ์ค€๋น„๋˜์—ˆ์œผ๋ฉด,\n # ํ…Œ์ด๋ธ”์˜ ํ–‰ ๋ฐ์ดํ„ฐ๋กœ ์ถ”๊ฐ€ํ•œ๋‹ค\n wb_table.add_data(row['filename'],\n image,\n row[\"xmin\"],\n row[\"ymin\"],\n row[\"xmax\"],\n row[\"ymax\"]\n )\n \nwandb.log({'Data Visualization': wb_table})\nrun.finish()","metadata":{"execution":{"iopub.status.busy":"2021-12-16T01:51:40.584526Z","iopub.execute_input":"2021-12-16T01:51:40.584843Z","iopub.status.idle":"2021-12-16T01:52:20.754018Z","shell.execute_reply.started":"2021-12-16T01:51:40.58481Z","shell.execute_reply":"2021-12-16T01:52:20.752946Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# ์ง์ ‘ W&B ๋Œ€์‹œ๋ณด๋“œ์—์„œ ํ™•์ธํ•ด๋„ ๋˜์ง€๋งŒ,\n# ์•„๋ž˜ ๋ช…๋ น์„ ์ˆ˜ํ–‰ํ•˜๋ฉด IFrame์œผ๋กœ W&B ๋Œ€์‹œ๋ณด๋“œ ์ธํ„ฐํŽ˜์ด์Šค๋ฅผ ์ฃผํ”ผํ„ฐ ๋…ธํŠธ๋ถ๋‚ด ์ถœ๋ ฅํ•œ๋‹ค\nframe = IFrame(run.url, width=1280, height=720)\nframe","metadata":{"execution":{"iopub.status.busy":"2021-12-16T01:52:22.062192Z","iopub.execute_input":"2021-12-16T01:52:22.062499Z","iopub.status.idle":"2021-12-16T01:52:22.074294Z","shell.execute_reply.started":"2021-12-16T01:52:22.062448Z","shell.execute_reply":"2021-12-16T01:52:22.073068Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n

Pandas Profiling

\n
\n

\n Pandas profiling is an open source Python module with which we can quickly do an exploratory data analysis with just a few lines of code. Besides, if this is not enough to convince us to use this tool, it also generates interactive reports in web format that can be presented to any person, even if they donโ€™t know programming.\n

\n Go to offocial website for documentation\n
","metadata":{}},{"cell_type":"code","source":"train_report = ProfileReport(df,title=\"Metadata of Training images\")\ntrain_report.to_file(\"./train_metadata.html\")\ntrain_report","metadata":{"execution":{"iopub.status.busy":"2021-12-16T01:52:57.440091Z","iopub.execute_input":"2021-12-16T01:52:57.440423Z","iopub.status.idle":"2021-12-16T01:53:16.366162Z","shell.execute_reply.started":"2021-12-16T01:52:57.440387Z","shell.execute_reply":"2021-12-16T01:53:16.363652Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n

IceVision

\n
\n

\n IceVision is the first agnostic computer vision framework to offer a curated collection with hundreds of high-quality pre-trained models from torchvision, MMLabs, and soon Pytorch Image Models. It orchestrates the end-to-end deep learning workflow allowing to train networks with easy-to-use robust high-performance libraries such as Pytorch-Lightning and Fastai

\n IceVision Unique Features:\n \n Go to offocial website for documentation\n
","metadata":{}},{"cell_type":"markdown","source":"

Modelling ๐Ÿซ

","metadata":{}},{"cell_type":"markdown","source":"

Data Parser

","metadata":{}},{"cell_type":"markdown","source":"How to create custom Parser for IceVision [[LINK](https://airctic.com/0.11.0/custom_parser/)]\n\n๋ฐ์ดํ„ฐ์…‹์„ ๋ชจ๋ธ์˜ ์ž…๋ ฅ์œผ๋กœ ๋งŒ๋“ค๊ธฐ์œ„ํ•œ ํด๋ž˜์Šค๋ฅผ ๋งŒ๋“ ๋‹ค. Pandas DataFrame์˜ ๋‚ด์šฉ์„ ํŒŒ์‹ฑํ•ด์•ผ ํ•˜๊ธฐ ๋•Œ๋ฌธ์—, Parser ๋ผ๊ณ  ๋ถˆ๋ฆฌ๋Š” ๊ฐ์ฒด๋ฅผ ํ™œ์šฉํ•˜๋ฉฐ, ๊ฐ์ฒด ํƒ์ง€์— ์‚ฌ์šฉ๋˜๋Š” ๊ธฐ๋ณธ ๊ฐ์ฒด๋Š” ObjectDetectionRecord ์ด๋‹ค.","metadata":{}},{"cell_type":"code","source":"# ๊ฐ์ฒด ํƒ์ง€๋ฅผ ์œ„ํ•œ ๋ ˆ์ฝ”๋“œ๋Š” ObjectDetectionRecord ๊ฐ์ฒด๋กœ ํ‘œํ˜„๋œ๋‹ค.\n# Parser.generate_template ๋ฉ”์„œ๋“œ๋Š” ๋ ˆ์ฝ”๋“œ ๊ฐ์ฒด๋ฅผ ๋งŒ๋“ค๊ธฐ ์œ„ํ•ด ํ•„์š”ํ•œ ๊ธฐ๋ณธ ํ…œํ”Œ๋ฆฟ์„ ์ถœ๋ ฅํ•œ๋‹ค.\ntemplate_record = ObjectDetectionRecord()\nParser.generate_template(template_record)","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:50:53.476792Z","iopub.execute_input":"2021-12-16T12:50:53.477114Z","iopub.status.idle":"2021-12-16T12:50:53.483971Z","shell.execute_reply.started":"2021-12-16T12:50:53.47708Z","shell.execute_reply":"2021-12-16T12:50:53.483058Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n# ๋ณด๋‹ค์‹œํ”ผ ์œ„์— ์ถœ๋ ฅ๋œ MyParser ํ…œํ”Œ๋ฆฟ์ฝ”๋“œ๋ฅผ ๊ทธ๋Œ€๋กœ ๊ฐ€์ ธ์˜จ ๋‹ค์Œ, ๊ฐ ๋ฉ”์„œ๋“œ๋ฅผ ๊ตฌํ˜„ํ•˜์˜€๋‹ค\nclass PGBRParser(Parser):\n def __init__(self, template_record, data_dir, df):\n super().__init__(template_record=template_record)\n\n self.data_dir = data_dir\n self.df = df\n self.class_map = ClassMap(list(self.df['label'].unique()))\n\n # ๊ฐ ๋ฐ์ดํ„ฐ ๋ ˆ์ฝ”๋“œ(ํ–‰)์„ ์ดํ„ฐ๋ ˆ์ด์…˜ํ•˜๊ณ , \n # ๊ฐ ์ดํ„ฐ๋ ˆ์ด์…˜ ๋งˆ๋‹ค ๋ฐ˜ํ™˜ํ•  ๊ฐ์ฒด๋ฅผ ์ •์˜ํ•œ๋‹ค\n def __iter__(self) -> Any:\n for o in self.df.itertuples():\n yield o\n\n def __len__(self) -> int:\n return len(self.df)\n\n # ์œ ๋‹ˆํฌ ID ํ• ๋‹น์— ์‚ฌ์šฉ๋  ์ˆ˜ ์žˆ๋Š” ์ •๋ณด๋ฅผ ๋ฐ˜ํ™˜ํ•œ๋‹ค.\n # ์—ฌ๊ธฐ์„œ o๋Š” __iter__๊ฐ€ ์ดํ„ฐ๋ ˆ์ด์…˜๋งˆ๋‹ค ๋ฐ˜ํ™˜ํ•˜๋Š” ๊ฐ์ฒด๋ฅผ ์˜๋ฏธํ•œ๋‹ค\n def record_id(self, o) -> Hashable:\n return o.filename\n\n # ์—ฌ๊ธฐ์„œ o๋Š” __iter__๊ฐ€ ์ดํ„ฐ๋ ˆ์ด์…˜๋งˆ๋‹ค ๋ฐ˜ํ™˜ํ•˜๋Š” ๊ฐ์ฒด๋ฅผ ์˜๋ฏธํ•œ๋‹ค\n #\n # ๋ณด๋‹ค์‹œํ”ผ is_new ๋ผ๋Š” ์ธ์ž๊ฐ€ ์กด์žฌํ•œ๋‹ค. ์‚ฌ์ง„ ํ•œ ์žฅ์— ์—ฌ๋Ÿฌ ๋ฌผ์ฒด๊ฐ€ ๊ธฐ๋ก๋˜์–ด ์žˆ์„ ์ˆ˜ ์žˆ๋‹ค.\n # DataFrame ํ–‰์„ ํ•˜๋‚˜์”ฉ ํƒ์ƒ‰ํ•˜๋‹ค๊ฐ€ ๋™์ผํ•œ ์ด๋ฏธ์ง€ ์ด๋ฆ„์„ ๊ฐ€์ง„ ๋ ˆ์ฝ”๋“œ(ํ–‰)์ด ๋˜ ๋‹ค์‹œ ๋ฐœ๊ฒฌ๋œ๋‹ค๋ฉด ์ด๋ฅผ ๊ฐ™์€ ๋ ˆ์ฝ”๋“œ๋กœ ๊ฐ„์ฃผํ•œ๋‹ค (record_id์—์„œ ์ˆ˜ํ–‰)\n # ๊ทธ๋ฆฌ๊ณ  record.detection.add_bboxes, record.detection.add_labels์— ์ถ”๊ฐ€ bbox ๋ฐ ๋ ˆ์ด๋ธ”์„ ๊ธฐ๋กํ•ด ๋„ฃ๋Š”๋‹ค.\n # is_new๋Š” ์ง€๊ธˆ๊นŒ์ง€ ๋ณธ ์  ์—†๋Š” record_id๊ฐ€ ๋ฐœ๊ฒฌ๋˜์—ˆ์„ ๋•Œ๋งŒ ์ˆ˜ํ–‰ํ•œ๋‹ค.\n #\n # ์ด ๋‚ด์šฉ์œผ๋กœ ๋ฏธ๋ฃจ์–ด ๋ณด๋ฉด, ํ•™์Šต ๋ฐ์ดํ„ฐ์˜ ์ž…๋ ฅ์ธ ์ด๋ฏธ์ง€๋Š” set_filepath๋กœ ์ง€์ •๋œ๋‹ค (์ด๋ฏธ์ง€ ๊ฐ์ฒด ์ž์ฒด๊ฐ€ ์•„๋‹ˆ๋‹ค)\n def parse_fields(self, o, record, is_new):\n if is_new:\n record.set_filepath(os.path.join(self.data_dir,o.filename))\n record.set_img_size(ImgSize(width=o.width, height=o.height))\n record.detection.set_class_map(self.class_map)\n\n record.detection.add_bboxes([BBox.from_xyxy(o.xmin, o.ymin, o.xmax, o.ymax)])\n record.detection.add_labels([o.label])","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:50:54.367468Z","iopub.execute_input":"2021-12-16T12:50:54.367738Z","iopub.status.idle":"2021-12-16T12:50:54.376448Z","shell.execute_reply.started":"2021-12-16T12:50:54.367708Z","shell.execute_reply":"2021-12-16T12:50:54.37545Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Data Loader

","metadata":{}},{"cell_type":"code","source":"# ๋ฐ์ดํ„ฐํ”„๋ ˆ์ž„์—์„œ ๋ฐ์ดํ„ฐ๋ฅผ ํŒŒ์‹ฑ (๋ ˆ์ด๋ธ”๊ณผ BBOX)\ntemplate_record = ObjectDetectionRecord()\nparser = PGBRParser(template_record, \"./dataset\", df)\n\ntrain_records, valid_records = parser.parse()\nprint(parser.class_map)\n# ์—ฌ๊ธฐ๊นŒ์ง€ํ•˜๋ฉด ํŒŒ์‹ฑ์ด ์™„๋ฃŒ๋œ๋‹ค.\n\ntrain_tfms = tfms.A.Adapter([*tfms.A.aug_tfms(size=image_size, presize=512), tfms.A.Normalize()])\nvalid_tfms = tfms.A.Adapter([*tfms.A.resize_and_pad(image_size), tfms.A.Normalize()])\n# ์—ฌ๊ธฐ๊นŒ์ง€ํ•˜๋ฉด ํŒŒ์‹ฑ๋œ ๋ฐ์ดํ„ฐ ์ ์šฉํ•  ๋ณ€ํ˜•๋“ค์˜ ์ •์˜๊ฐ€ ์™„๋ฃŒ๋œ๋‹ค.\n\ntrain_ds = Dataset(train_records, train_tfms)\nvalid_ds = Dataset(valid_records, valid_tfms)\n# ์—ฌ๊ธฐ๊นŒ์ง€ ํ•˜๋ฉด ์‹ค์ œ ๋ชจ๋ธ์— ์ž…๋ ฅ๋  Dataset ๊ฐ์ฒด ์ƒ์„ฑ์ด ์™„๋ฃŒ๋œ๋‹ค.\n# ๋ณด๋‹ค์‹œํ”ผ ์ผ๋ถ€ ๋ฐ์ดํ„ฐ ์ฆ๊ฐ•๊นŒ์ง€ ์ˆ˜ํ–‰๋˜์—ˆ๋‹ค. (presizing - https://notesbylex.com/presizing.html)\n\nsamples = [train_ds[0] for _ in range(3)]\nshow_samples(samples, ncols=3)\n# ์ผ๋ถ€ ๋ฐ์ดํ„ฐ๋ฅผ ์ถœ๋ ฅํ•œ๋‹ค","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:51:04.913537Z","iopub.execute_input":"2021-12-16T12:51:04.914179Z","iopub.status.idle":"2021-12-16T12:51:10.195602Z","shell.execute_reply.started":"2021-12-16T12:51:04.914128Z","shell.execute_reply":"2021-12-16T12:51:10.194845Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# model_type์€ ๊ฐ์ฒดํƒ์ง€์šฉ ์ „์ฒด ๋ชจ๋ธ ๊ตฌ์กฐ๋ฅผ ์„ ํƒํ•œ๋‹ค\n# - https://velog.io/@zeen263/%ED%94%84%EB%A1%9C%EC%A0%9D%ED%8A%B8-3-Detection-2.-VFNet-%EB%85%BC%EB%AC%B8-%EB%A6%AC%EB%B7%B0\nmodel_type = models.mmdet.vfnet\n\n# ๋ชจ๋ธ ๊ตฌ์กฐ ์ค‘ ๋ฐฑ๋ณธ์œผ๋กœ ์‚ฌ์šฉํ•  ๋ชจ๋ธ์„ ์„ ํƒํ•œ๋‹ค\nbackbone = model_type.backbones.resnet50_fpn_mstrain_2x(pretrained=True)\n\n# ๊ทธ๋ฆฌ๊ณ  ์ด ๋‘˜์„ ๊ฒฐํ•จํ•ด ์ „์ฒด ๋ชจ๋ธ์˜ ์ธ์Šคํ„ด์Šค๋ฅผ ์ƒ์„ฑํ•œ๋‹ค\nmodel = model_type.model(backbone=backbone, num_classes=len(parser.class_map))\n\n# ๋ชจ๋ธ์— ์ž…๋ ฅ๋  ๋ฐ์ดํ„ฐ๋ฅผ ์ •์˜ํ•˜๋Š” DataLoader ๊ฐ์ฒด๋ฅผ ์ƒ์„ฑํ•œ๋‹ค\ntrain_dl = model_type.train_dl(train_ds, batch_size=16, num_workers=4, shuffle=True)\nvalid_dl = model_type.valid_dl(valid_ds, batch_size=16, num_workers=4, shuffle=False)","metadata":{"_kg_hide-output":true,"scrolled":true,"execution":{"iopub.status.busy":"2021-12-16T01:54:13.096071Z","iopub.execute_input":"2021-12-16T01:54:13.0967Z","iopub.status.idle":"2021-12-16T01:54:50.48565Z","shell.execute_reply.started":"2021-12-16T01:54:13.09665Z","shell.execute_reply":"2021-12-16T01:54:50.484514Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Metrics

","metadata":{}},{"cell_type":"code","source":"# VFNet์ด ํƒ€๊ฒŒํŒ…ํ•œ ๋ฐ์ดํ„ฐ์…‹์€ COCO ์ด๋‹ค\nmetrics = [COCOMetric(metric_type=COCOMetricType.bbox)]","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:56:04.523599Z","iopub.execute_input":"2021-12-16T12:56:04.52389Z","iopub.status.idle":"2021-12-16T12:56:04.528597Z","shell.execute_reply.started":"2021-12-16T12:56:04.523855Z","shell.execute_reply":"2021-12-16T12:56:04.52783Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Training

","metadata":{}},{"cell_type":"code","source":"learn = model_type.fastai.learner(dls=[train_dl, valid_dl], model=model, metrics=metrics)","metadata":{"execution":{"iopub.status.busy":"2021-12-16T01:55:44.478828Z","iopub.execute_input":"2021-12-16T01:55:44.479184Z","iopub.status.idle":"2021-12-16T01:55:44.841406Z","shell.execute_reply.started":"2021-12-16T01:55:44.479149Z","shell.execute_reply":"2021-12-16T01:55:44.840263Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# learn.lr_find()","metadata":{"execution":{"iopub.status.busy":"2021-12-15T14:02:51.872173Z","iopub.execute_input":"2021-12-15T14:02:51.872525Z","iopub.status.idle":"2021-12-15T14:04:54.539914Z","shell.execute_reply.started":"2021-12-15T14:02:51.872486Z","shell.execute_reply":"2021-12-15T14:04:54.539137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.fine_tune(30, 0.0003, freeze_epochs=5)","metadata":{"execution":{"iopub.status.busy":"2021-12-16T01:55:51.16381Z","iopub.execute_input":"2021-12-16T01:55:51.16415Z","iopub.status.idle":"2021-12-16T06:36:23.723767Z","shell.execute_reply.started":"2021-12-16T01:55:51.164107Z","shell.execute_reply":"2021-12-16T06:36:23.722544Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"torch.save(model.state_dict(), './gbr_vfnet_resnet50_fpn_mstrain_2x_30epochs.pt')","metadata":{"execution":{"iopub.status.busy":"2021-12-16T06:41:10.96571Z","iopub.execute_input":"2021-12-16T06:41:10.966058Z","iopub.status.idle":"2021-12-16T06:41:11.254012Z","shell.execute_reply.started":"2021-12-16T06:41:10.966026Z","shell.execute_reply":"2021-12-16T06:41:11.252829Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls ../input/gbr-vfnet-resnet50-fpn-mstrain-2x-30epochspt/gbr_vfnet_resnet50_fpn_mstrain_2x_30epochs.pt","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:51:25.127727Z","iopub.execute_input":"2021-12-16T12:51:25.128025Z","iopub.status.idle":"2021-12-16T12:51:25.833698Z","shell.execute_reply.started":"2021-12-16T12:51:25.127991Z","shell.execute_reply":"2021-12-16T12:51:25.832754Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"state_dict = torch.load('../input/gbr-vfnet-resnet50-fpn-mstrain-2x-30epochspt/gbr_vfnet_resnet50_fpn_mstrain_2x_30epochs.pt')","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:53:05.105276Z","iopub.execute_input":"2021-12-16T12:53:05.105599Z","iopub.status.idle":"2021-12-16T12:53:16.452642Z","shell.execute_reply.started":"2021-12-16T12:53:05.105564Z","shell.execute_reply":"2021-12-16T12:53:16.4518Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_type = models.mmdet.vfnet\nbackbone = model_type.backbones.resnet50_fpn_mstrain_2x(pretrained=True)\nmodel = model_type.model(backbone=backbone, num_classes=len(parser.class_map))\n\ntrain_dl = model_type.train_dl(train_ds, batch_size=16, num_workers=4, shuffle=True)\nvalid_dl = model_type.valid_dl(valid_ds, batch_size=16, num_workers=4, shuffle=False)\n\nmodel.load_state_dict(state_dict)","metadata":{"_kg_hide-output":false,"execution":{"iopub.status.busy":"2021-12-16T12:53:21.773699Z","iopub.execute_input":"2021-12-16T12:53:21.774114Z","iopub.status.idle":"2021-12-16T12:53:43.131766Z","shell.execute_reply.started":"2021-12-16T12:53:21.774062Z","shell.execute_reply":"2021-12-16T12:53:43.130872Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn = model_type.fastai.learner(dls=[train_dl, valid_dl], model=model, metrics=metrics)","metadata":{"execution":{"iopub.status.busy":"2021-12-16T12:56:18.631493Z","iopub.execute_input":"2021-12-16T12:56:18.631766Z","iopub.status.idle":"2021-12-16T12:56:18.657451Z","shell.execute_reply.started":"2021-12-16T12:56:18.631734Z","shell.execute_reply":"2021-12-16T12:56:18.656645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.fine_tune(1, 10e-5, freeze_epochs=0)","metadata":{"execution":{"iopub.status.busy":"2021-12-16T13:02:42.512468Z","iopub.execute_input":"2021-12-16T13:02:42.512774Z","iopub.status.idle":"2021-12-16T13:10:01.032692Z","shell.execute_reply.started":"2021-12-16T13:02:42.512738Z","shell.execute_reply":"2021-12-16T13:10:01.031865Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Inference ๐Ÿ”ฎ

","metadata":{}},{"cell_type":"code","source":"infer_dl = model_type.infer_dl([valid_ds[0],valid_ds[4],valid_ds[7],valid_ds[9]], batch_size=4, shuffle=False)\npreds = model_type.predict_from_dl(model, infer_dl, keep_images=True)\nshow_preds(preds=preds)","metadata":{"execution":{"iopub.status.busy":"2021-12-16T13:24:07.718007Z","iopub.execute_input":"2021-12-16T13:24:07.718317Z","iopub.status.idle":"2021-12-16T13:24:10.099679Z","shell.execute_reply.started":"2021-12-16T13:24:07.718282Z","shell.execute_reply":"2021-12-16T13:24:10.098895Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# shutil.make_archive('./checkpoints',\n# 'zip',\n# './',\n# 'checkpoints')\n\n# shutil.rmtree(\"./checkpoints\")\n# shutil.rmtree(\"./dataset\")\n# shutil.rmtree(\"./wandb\")","metadata":{"execution":{"iopub.status.busy":"2021-12-13T21:22:40.848691Z","iopub.execute_input":"2021-12-13T21:22:40.849147Z","iopub.status.idle":"2021-12-13T21:22:57.503833Z","shell.execute_reply.started":"2021-12-13T21:22:40.84911Z","shell.execute_reply":"2021-12-13T21:22:57.502789Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":" Download File ","metadata":{}},{"cell_type":"markdown","source":"
\n

If you come this far, you could've got some insights from this notebook. An upvote would be very helpful :). Kindly comment if there are any doubts or mistakes

\n\n
","metadata":{}}]}