{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"![](https://i.pinimg.com/originals/1e/0a/8c/1e0a8c05fa4735cbfc94c8fa337b305c.gif)","metadata":{}},{"cell_type":"markdown","source":"
\n ๐Ÿ’Š Medical Instance Segmentation with YOLOv8\n
Hi all! ๐Ÿ‘จโ€โš•๏ธ
\n Today we will dive into the problem of segmentation of instances in a medical context, and we will also use the YOLOv8 model, which is SOTA today. I hope you can learn something new.\n
","metadata":{}},{"cell_type":"markdown","source":"![](https://i.pinimg.com/originals/62/3e/2f/623e2feebd36f364f704865c7603249d.gif)","metadata":{}},{"cell_type":"markdown","source":"
\n Biological overview โš•๏ธ\n
Our task is to segment medical images of kidney tissue samples. Human tissues are covered with many vessels. Our task is to segment such vessels.
\n
","metadata":{}},{"cell_type":"markdown","source":"![](https://i.ibb.co/1MZNhNF/003.png)","metadata":{}},{"cell_type":"markdown","source":"
\n Tissue samples ๐Ÿงช\n
Renal cortex: the renal cortex is the outer portion of the kidney that contains round renal corpuscles enclosing glomerular tufts, balls of capillary loops. The renal corpuscle is the start of the nephron, through which the filtration of blood occurs. The renal cortex also contains proximal and distal convoluted tubules, PCTs and DCTs, which are also regions of the nephron. Between these tubular structures is a complex network of capillaries called the peritubular capillaries.
\n
Renal medulla: the renal medulla is the inner portion of the kidney and is arranged in 8-15 renal pyramids containing linearly arranged tubules comprising the loops of Henle and ducts that gather products for excretion. The capillary network in the renal medulla consists of capillaries called vasa recta. The renal pyramids (medullary tissue) are divided by extensions of the renal cortex called renal columns. There are also projections of the renal medulla into the outer cortex called medullary rays.
\n
Renal papilla (subsection of medulla): the broad bases of the pyramids connect to the renal cortex at the corticomedullary junctions while the tips form structures called the renal papilla, which project in the minor renal calyces where urine is collected.
\n
","metadata":{}},{"cell_type":"markdown","source":"
\n What is instance segmentation?\n
Instance Segmentation is a unique form of image segmentation that deals with detecting and delineating each distinct instance of an object appearing in an image. Instance segmentation detects all instances of a class with the extra functionality of demarcating separate instances of any segment class. Hence, it is also referred to as incorporating object detection and semantic segmentation functionality. Watch video to learn more: Instance segmentation | Tutorial
\n
","metadata":{}},{"cell_type":"markdown","source":"
\n Applications of Semantic Segmentation\n
  1. Medical Diagnostics: For detecting medical abnormalities in X-Rays, CT Scans, MRI Scans
  2. GeoSensing: For land usage mapping from satellite imagery and monitoring areas of deforestation and urbanization
  3. Autonomous Driving: For accurately detecting lanes, pedestrians, traffic signs, road, sky and other vehicles on the road

\n
\n\n","metadata":{}},{"cell_type":"markdown","source":"# Prepare data\n
\n It's time to think about data ๐Ÿ“š\n
For training, we need a dataset, and not just any, but in the COCO format. COCO is an open database for object detection, it is huge and therefore it is used all over the world to evaluate new models according to various metrics. It plays a rather important role, and most detection models are pre-trained on it. That is why this dataset format is used in ultralytics. You don't need to worry about this, as I have already prepared a class for data processing. I advise you to skip the cell below and immediately proceed to the next chapter.
\n
","metadata":{}},{"cell_type":"markdown","source":"
\n Libraries for creating dataset\n
","metadata":{}},{"cell_type":"code","source":"from itertools import chain\nimport json\nimport os\nimport shutil\nfrom tqdm.notebook import tqdm\nfrom colorama import Fore\nimport yaml\nimport numpy as np","metadata":{"execution":{"iopub.status.busy":"2023-08-10T20:35:35.412226Z","iopub.execute_input":"2023-08-10T20:35:35.412650Z","iopub.status.idle":"2023-08-10T20:35:35.488377Z","shell.execute_reply.started":"2023-08-10T20:35:35.412614Z","shell.execute_reply":"2023-08-10T20:35:35.487408Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n Class for creating dataset\n
","metadata":{}},{"cell_type":"code","source":"class COCODataset:\n def __init__(self, images_dirpath: str, annotations_filepath: str, length: int = 1633):\n self.train_size = None\n self.val_size = None\n self.length = length\n self.classes = None\n self.labels_counter = None\n self.normalize = None\n \n self.images_dirpath = images_dirpath\n self.annotations_filepath = annotations_filepath\n self.dataset_dirpath = os.path.join(os.getcwd(), \"dataset\")\n self.train_dirpath = os.path.join(self.dataset_dirpath, \"train\")\n self.val_dirpath = os.path.join(self.dataset_dirpath, \"val\")\n self.config_path = os.path.join(self.dataset_dirpath, \"coco.yaml\")\n\n self.samples = self.parse_jsonl(annotations_filepath)\n self.classes_dict = {\n \"blood_vessel\": 0,\n \"glomerulus\": 1,\n \"unsure\": 2,\n }\n\n def __prepare_dirs(self) -> None:\n os.makedirs(os.path.join(self.train_dirpath, \"images\"), exist_ok=True)\n os.makedirs(os.path.join(self.train_dirpath, \"labels\"), exist_ok=True)\n os.makedirs(os.path.join(self.val_dirpath, \"images\"), exist_ok=True)\n os.makedirs(os.path.join(self.val_dirpath, \"labels\"), exist_ok=True)\n \n\n def __define_splitratio(self) -> None:\n self.train_size = round(self.length * self.train_size)\n self.val_size = self.length - self.train_size\n assert self.train_size + self.val_size == self.length\n\n def parse_jsonl(self, path: str) -> list[dict, ...]:\n with open(path, 'r') as json_file:\n jsonl_samples = [\n json.loads(line)\n for line in tqdm(\n json_file, desc=\"Processing polygons\", total=self.length\n )\n ]\n return jsonl_samples\n\n def __define_paths(self, i: int) -> dict:\n data_path = self.val_dirpath\n if i < self.train_size:\n data_path = self.train_dirpath\n return {\n \"images\": os.path.join(data_path, \"images\"),\n \"labels\": os.path.join(data_path, \"labels\")\n }\n \n \n @staticmethod\n def __get_label_path(paths_dict: dict, identifier: str) -> str:\n return os.path.join(\n paths_dict[\"labels\"],\n f\"{identifier}.txt\"\n )\n\n @staticmethod\n def __get_image_path(paths_dict: dict, identifier: str) -> str:\n return os.path.join(\n paths_dict[\"images\"],\n f\"{identifier}.tif\"\n )\n\n def __copy_image(self, dst_path: str, identifier: str) -> str:\n shutil.copyfile(\n os.path.join(self.images_dirpath, f\"{identifier}.tif\"),\n dst_path\n )\n\n def __copy_label(self, annotations: list, dst_path: str) -> None:\n with open(dst_path, \"w\") as file:\n for annotation in annotations:\n coordinates = annotation[\"coordinates\"][0]\n label = self.classes_dict[annotation[\"type\"]]\n if label in self.classes:\n if coordinates:\n if self.normalize:\n coordinates = np.array(coordinates) / 512.0\n coordinates = \" \".join(map(str, chain(*coordinates)))\n file.write(f\"{label} {coordinates}\\n\")\n self.labels_counter += 1\n\n def __splitfolders(self):\n for i, line in tqdm(\n enumerate(self.samples),\n desc=\"Dataset creation\", total=self.length\n ):\n self.labels_counter = 0\n identifier = line[\"id\"]\n annotations = line[\"annotations\"]\n paths_dict = self.__define_paths(i)\n\n dst_image_path = self.__get_image_path(paths_dict, identifier)\n dst_label_path = self.__get_label_path(paths_dict, identifier)\n\n self.__copy_image(dst_image_path, identifier)\n self.__copy_label(annotations, dst_label_path)\n\n if self.labels_counter == 0:\n os.remove(dst_image_path)\n os.remove(dst_label_path)\n\n def __count_dataset(self) -> dict:\n train_images = len(os.listdir(os.path.join(self.train_dirpath, \"images\")))\n train_labels = len(os.listdir(os.path.join(self.train_dirpath, \"labels\")))\n val_images = len(os.listdir(os.path.join(self.val_dirpath, \"images\")))\n val_labels = len(os.listdir(os.path.join(self.val_dirpath, \"labels\")))\n return {\n \"train_images\": train_images,\n \"train_labels\": train_labels,\n \"val_images\": val_images,\n \"val_labels\": val_labels\n }\n\n @staticmethod\n def __check_sanity(count_dict: dict) -> None:\n assert count_dict[\"train_images\"] == count_dict[\"train_labels\"]\n assert count_dict[\"val_images\"] == count_dict[\"val_labels\"]\n\n def __finalizing(self, count_dict: dict) -> None:\n assert os.path.exists(self.dataset_dirpath)\n\n example_structure = [\n \"dataset\",\n \"train\", \"labels\", \"images\",\n \"val\", \"labels\", \"images\"\n ]\n\n dir_bone = (\n dirname.split(\"/\")[-1]\n for dirname, _, filenames in os.walk(self.dataset_dirpath)\n if dirname.split(\"/\")[-1] in example_structure\n )\n\n try:\n print(\"\\n~ HuBMAP Dataset Structure ~\\n\")\n print(\n f\"\"\"\n โ”œโ”€โ”€ {next(dir_bone)}\n โ”‚ โ”‚\n โ”‚ โ”œโ”€โ”€ {next(dir_bone)}\n โ”‚ โ”‚ โ””โ”€โ”€ {next(dir_bone)}\n โ”‚ โ”‚ โ””โ”€โ”€ {next(dir_bone)}\n โ”‚ โ”‚\n โ”‚ โ”œโ”€โ”€ {next(dir_bone)}\n โ”‚ โ”‚ โ””โ”€โ”€ {next(dir_bone)}\n โ”‚ โ”‚ โ””โ”€โ”€ {next(dir_bone)}\n \"\"\"\n )\n except StopIteration as e:\n print(e)\n else:\n print(Fore.GREEN + \"-> Success\")\n print(Fore.GREEN + f\"Train dataset: {count_dict['train_images']}\\nVal dataset: {count_dict['val_images']}\")\n\n def get_config(self) ->dict:\n names = [\"blood_vessel\", \"glomerulus\", \"unsure\"]\n return {\n \"train\": str(self.train_dirpath),\n \"val\": str(self.val_dirpath),\n \"names\": [names[i] for i in self.classes]\n }\n\n @staticmethod\n def display_config(config: dict) -> None:\n print(Fore.BLACK + \"\\n~ HuBMAP Config Structure ~\\n\")\n print(\n f\"\"\"\n โ”‚ โ”‚\n โ”‚ โ”œโ”€โ”€ train\n โ”‚ โ”‚ โ””โ”€โ”€ {config['train']}/images\n โ”‚ โ”‚\n โ”‚ โ”‚\n โ”‚ โ”œโ”€โ”€ val\n โ”‚ โ”‚ โ””โ”€โ”€ {config['val']}/images\n โ”‚ โ”‚\n โ”‚ โ”‚\n โ”‚ โ”œโ”€โ”€ names\n โ”‚ โ”‚ โ””โ”€โ”€ {' '.join(config['names'])}\n \"\"\"\n )\n print(Fore.GREEN + \"-> Success\")\n print(Fore.GREEN + f\"Number of classes: {len(config['names'])}\"\n f\"\\nClasses: {' '.join(config['names'])}\" \n )\n\n def write_config(self, config: dict) -> None:\n with open(self.config_path, mode=\"w\") as f:\n yaml.safe_dump(stream=f, data=config)\n\n def __call__(self, train_size: float,\n classes: list[int, ...],\n make_config: bool = True,\n normalize: bool = True\n ) -> None:\n self.train_size = train_size\n self.classes = classes\n self.normalize = normalize\n \n self.__define_splitratio()\n self.__prepare_dirs()\n self.__splitfolders()\n count_dict = self.__count_dataset()\n self.__check_sanity(count_dict)\n self.__finalizing(count_dict)\n \n if make_config:\n config = self.get_config()\n self.write_config(config)\n self.display_config(config)","metadata":{"execution":{"iopub.status.busy":"2023-08-10T20:36:23.963304Z","iopub.execute_input":"2023-08-10T20:36:23.963684Z","iopub.status.idle":"2023-08-10T20:36:24.007294Z","shell.execute_reply.started":"2023-08-10T20:36:23.963651Z","shell.execute_reply":"2023-08-10T20:36:24.006297Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Defining another dataset used for visualization","metadata":{}},{"cell_type":"code","source":"from torch.utils.data import Dataset\nfrom tqdm import tqdm, notebook \n\nclass HuBMAPDataset(Dataset):\n def __init__(self, \n annotation_path: str,\n image_path: str, \n config_path: str):\n self.__image_path = image_path\n self.__samples = self.parse_jsonl(annotation_path)\n self.__config = self.load_config(config_path)\n \n def __len__(self) -> int:\n return len(self.__samples)\n\n def __getitem__(self, idx: int) -> tuple[np.ndarray, np.ndarray]:\n id, image = self.__get_image(idx)\n mask = self.__get_mask(idx)\n # image = torch.tensor(image, dtype=torch.float32).permute(2, 0, 1)\n # mask = torch.tensor(mask, dtype=torch.float32) \n return id, image, mask\n \n @staticmethod\n def parse_jsonl(path: str) -> list[dict, ...]:\n with open(path, 'r') as json_file:\n jsonl_labels = [\n json.loads(line)\n for line in notebook.tqdm(\n json_file, desc=\"Processing polygons\", total=1633\n )\n ]\n return jsonl_labels\n\n @staticmethod\n def load_config(path: str) -> dict:\n with open(path, mode=\"r\") as f:\n data = yaml.load(stream=f, Loader=yaml.SafeLoader)\n return data\n \n def __get_image_path(self, id: str) -> str:\n path = os.path.join(\n self.__image_path, f\"{id}.tif\"\n )\n return path\n \n def __get_image(self, idx: int) -> np.ndarray:\n image_path = self.__get_image_path(id)\n image = Image.open(image_path)\n image = np.asarray(image)\n return image\n \n def get_image_and_mask_by_id(self, id: str):\n image_path = self.__get_image_path(id)\n image = Image.open(image_path)\n image = np.asarray(image)\n mask = None\n for i, sample in enumerate(self.__samples):\n if sample['id'] == id:\n mask = self.__get_mask(i)\n break # Exit the loop once the mask is found\n if mask is not None: \n return image, mask\n else:\n raise FileNotFoundError(f\"Mask not found for ID: {id}\")\n \n def __get_mask(self, idx: int) -> np.ndarray:\n mask = np.zeros((512, 512), dtype=np.uint8)\n annotations = self.__samples[idx][\"annotations\"]\n \n for vessel in annotations:\n vessel_type = vessel[\"type\"] \n config = self.__config[vessel_type]\n \n if config[\"apply_mask\"]:\n coordinates = np.array(vessel[\"coordinates\"])\n mask = cv2.fillPoly(\n mask, pts=coordinates,\n color=config[\"rgb\"]\n )\n return mask","metadata":{"execution":{"iopub.status.busy":"2023-08-10T20:46:30.095008Z","iopub.execute_input":"2023-08-10T20:46:30.095390Z","iopub.status.idle":"2023-08-10T20:46:30.112701Z","shell.execute_reply.started":"2023-08-10T20:46:30.095360Z","shell.execute_reply":"2023-08-10T20:46:30.111803Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import cv2\nimport yaml\nimport json\nfrom PIL import Image\nimport matplotlib.pyplot as plt \n\ndataset_config = {\n \"background\": {\n \"apply_mask\": None,\n \"label\": 0,\n \"rgb\": (0, 0, 0),\n \"loss_weight\": None\n },\n \"blood_vessel\": {\n \"apply_mask\": True,\n \"label\": 1,\n \"rgb\": (255, 8, 8),\n \"loss_weight\": None\n },\n \"glomerulus\": {\n \"apply_mask\": True,\n \"label\": 2,\n \"rgb\": (8, 12, 255),\n \"loss_weight\": None\n },\n \"unsure\": {\n \"apply_mask\": True,\n \"label\": 3,\n \"rgb\": (8, 255, 20),\n \"loss_weight\": None\n }\n}","metadata":{"execution":{"iopub.status.busy":"2023-08-10T20:50:28.016260Z","iopub.execute_input":"2023-08-10T20:50:28.016623Z","iopub.status.idle":"2023-08-10T20:50:28.023496Z","shell.execute_reply.started":"2023-08-10T20:50:28.016592Z","shell.execute_reply":"2023-08-10T20:50:28.022574Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Files \n__SAMPLE_SBMISSION_PATH = \"/kaggle/input/hubmap-hacking-the-human-vasculature/sample_submission.csv\"\n__TILE_META_PATH = \"/kaggle/input/hubmap-hacking-the-human-vasculature/tile_meta.csv\"\n__WSI_META_PATH = \"/kaggle/input/hubmap-hacking-the-human-vasculature/wsi_meta.csv\"\n__ANNOTATION_PATH = \"/kaggle/input/hubmap-hacking-the-human-vasculature/polygons.jsonl\"\n__DATASET_CONFIG_PATH = \"/kaggle/working/classes_config.csv\" # Custom file\n# Folders\n__TRAIN_PATH = \"/kaggle/input/hubmap-hacking-the-human-vasculature/train\"\n__TEST_PATH = \"/kaggle/input/hubmap-hacking-the-human-vasculature/test\"\ndef write_config(data: dict[dict, ...], path: str) -> None:\n with open(path, mode=\"w\") as f:\n yaml.safe_dump(stream=f, data=data)\n \nwrite_config(dataset_config, __DATASET_CONFIG_PATH)","metadata":{"execution":{"iopub.status.busy":"2023-08-10T20:46:47.620697Z","iopub.execute_input":"2023-08-10T20:46:47.621076Z","iopub.status.idle":"2023-08-10T20:46:47.632352Z","shell.execute_reply.started":"2023-08-10T20:46:47.621044Z","shell.execute_reply":"2023-08-10T20:46:47.631273Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dataset = HuBMAPDataset(__ANNOTATION_PATH, __TRAIN_PATH, __DATASET_CONFIG_PATH)","metadata":{"execution":{"iopub.status.busy":"2023-08-10T20:48:32.070830Z","iopub.execute_input":"2023-08-10T20:48:32.071226Z","iopub.status.idle":"2023-08-10T20:48:35.452135Z","shell.execute_reply.started":"2023-08-10T20:48:32.071193Z","shell.execute_reply":"2023-08-10T20:48:35.451165Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dirlist = os.listdir(\"/kaggle/working/dataset/val/images\")\nids = [file.split(\".\")[0] for file in dirlist]\nprint(ids[:5])","metadata":{"execution":{"iopub.status.busy":"2023-08-10T20:53:27.681549Z","iopub.execute_input":"2023-08-10T20:53:27.681957Z","iopub.status.idle":"2023-08-10T20:53:27.689067Z","shell.execute_reply.started":"2023-08-10T20:53:27.681923Z","shell.execute_reply":"2023-08-10T20:53:27.688040Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for id in ids[:2]:\n image, mask = dataset.get_image_and_mask_by_id(id)\n fig, ax = plt.subplots(1, 1)\n plt.title(f\"Segmentation for {id}\")\n # Display the image\n ax.imshow(image, cmap='gray') # You may want to use a different colormap for the image\n\n # Display the mask with a different colormap and some transparency\n ax.imshow(mask, cmap='jet', alpha=0.5) # You can adjust the alpha for desired transparency\n\n plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-08-10T20:53:28.911207Z","iopub.execute_input":"2023-08-10T20:53:28.912338Z","iopub.status.idle":"2023-08-10T20:53:29.710866Z","shell.execute_reply.started":"2023-08-10T20:53:28.912292Z","shell.execute_reply":"2023-08-10T20:53:29.710027Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n Let's fire up the data ๐Ÿ”ฅ\n
","metadata":{}},{"cell_type":"code","source":"coco = COCODataset(\n annotations_filepath=\"/kaggle/input/hubmap-hacking-the-human-vasculature/polygons.jsonl\",\n images_dirpath=\"/kaggle/input/hubmap-hacking-the-human-vasculature/train\",\n) ","metadata":{"execution":{"iopub.status.busy":"2023-08-10T20:36:25.920346Z","iopub.execute_input":"2023-08-10T20:36:25.920700Z","iopub.status.idle":"2023-08-10T20:36:29.758896Z","shell.execute_reply.started":"2023-08-10T20:36:25.920671Z","shell.execute_reply":"2023-08-10T20:36:29.757805Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"coco(train_size=0.80, classes=[0, 1, 2])","metadata":{"execution":{"iopub.status.busy":"2023-08-10T20:36:40.148546Z","iopub.execute_input":"2023-08-10T20:36:40.148916Z","iopub.status.idle":"2023-08-10T20:36:54.662566Z","shell.execute_reply.started":"2023-08-10T20:36:40.148887Z","shell.execute_reply":"2023-08-10T20:36:54.661652Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n And how can I use the right tools if the internet is banned in the competition? ๐Ÿ™\n
In the competition, many have difficulty installing the necessary tools and using them, because the Internet is prohibited, but it is prohibited only so that the test data is not stolen (they are uploaded to the test folder during the submission of the result). This means that you can use the tools you want, but you must add them as data. You can find my ultralytics and pycocotools dataset here. Importantly, the Internet is still present in this notebook to load the model, however, after training it, you will save the model, add it to the data and easily use it for forecasting without the Internet.
\n
","metadata":{}},{"cell_type":"code","source":"import shutil\nimport os\nimport sys\nfrom colorama import Fore","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class SetupPipline:\n def __init__(self, display: bool = True):\n self.pycocotools = self.__pycocotools()\n self.ultralytics = self.__ultralytics()\n \n @staticmethod\n def __ultralytics() -> str:\n sys.path.append(\"/kaggle/input/hubmap-tools-ultralytics-and-pycocotools/ultralytics/ultralytics\") \n return \"successfully\"\n \n @staticmethod\n def __pycocotools() -> str:\n if not os.path.exists(\"/kaggle/working/packages\"):\n shutil.copytree(\"/kaggle/input/hubmap-tools-ultralytics-and-pycocotools/pycocotools/pycocotools\", \"/kaggle/working/packages\")\n os.chdir(\"/kaggle/working/packages/pycocotools-2.0.6/\")\n os.system(\"python setup.py install\")\n os.system(\"pip install . --no-index --find-links /kaggle/working/packages/\")\n os.chdir(\"/kaggle/working\")\n return \"successfully\"\n \n def display(self) -> None:\n print(Fore.GREEN+f\"\\nPycocotools was installed {self.pycocotools}\")\n print(f\"Ultralytics was installed {self.ultralytics}\"+Fore.WHITE)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pipline = SetupPipline()","metadata":{"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pipline.display()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from pycocotools import _mask as coco_mask \nfrom ultralytics import YOLO","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Logging\n
\n And now a little about the control of experiments\n
Experiment control tools are a very important part of your pipeline. With the help of them, you can track learning and compare results with previous ones to solve your problem. It is very convenient and allows you not to get confused.
\n
","metadata":{}},{"cell_type":"markdown","source":"# ClearML Logging and Automation ๐ŸŒŸ NEW\n\n[ClearML](https://cutt.ly/yolov5-notebook-clearml) is completely integrated into YOLOv5 to track your experimentation, manage dataset versions and even remotely execute training runs. To enable ClearML (check cells above):\n\n- `pip install clearml`\n- run `clearml-init` to connect to a ClearML server (**deploy your own [open-source server](https://github.com/allegroai/clearml-server)**, or use our [free hosted server](https://cutt.ly/yolov5-notebook-clearml))\n\nYou'll get all the great expected features from an experiment manager: live updates, model upload, experiment comparison etc. but ClearML also tracks uncommitted changes and installed packages for example. Thanks to that ClearML Tasks (which is what we call experiments) are also reproducible on different machines! With only 1 extra line, we can schedule a YOLOv5 training task on a queue to be executed by any number of ClearML Agents (workers).\n\nYou can use ClearML Data to version your dataset and then pass it to YOLOv5 simply using its unique ID. This will help you keep track of your data without adding extra hassle. Explore the [ClearML Tutorial](https://github.com/ultralytics/yolov5/tree/master/utils/loggers/clearml) for details!\n\n\n\"ClearML","metadata":{}},{"cell_type":"markdown","source":"# Weights and biases\n \n

\n \n \n \n \n

\n

\n \n

\n \n Use W&B to build better models faster. Track and visualize all the pieces of your machine learning pipeline, from datasets to production machine learning models. Get started with W&B today, [sign up for a free account!](https://wandb.com?utm_source=github&utm_medium=code&utm_campaign=wandb&utm_content=readme) \n \n ๐ŸŽ“ W&B is free for students, educators, and academic researchers. For more information, visit [https://wandb.ai/site/research](https://wandb.ai/site/research?utm_source=github&utm_medium=code&utm_campaign=wandb&utm_content=readme). \n \n Want to use Weights & Biases for seamless collaboration between your ML or Data Science team? Looking for Production-grade MLOps at scale? Sign up to one of [our plans](https://wandb.ai/site/pricing) or [contact the Sales Team](https://wandb.ai/site/contact).\n","metadata":{}},{"cell_type":"markdown","source":"# YOLOv8 Architecture: A Deep Dive\n
\nYOLOv8 does not yet have a published paper, so we lack direct insight into the direct research methodology and ablation studies done during its creation. With that said, we analyzed the repository and information available about the model to start documenting what's new in YOLOv8.\n\nIf you want to peer into the code yourself, check out the YOLOv8 repository and you view this code differential to see how some of the research was done.\n\nHere we provide a quick summary of impactful modeling updates and then we will look at the model's evaluation, which speaks for itself.\n\nThe following image made by GitHub user RangeKing shows a detailed visualisation of the network's architecture.\n
","metadata":{}},{"cell_type":"markdown","source":"![](https://blog.roboflow.com/content/images/size/w1000/2023/01/image-16.png)","metadata":{}},{"cell_type":"markdown","source":"# Training\n
\n Finally, we can start learning ๐Ÿฅณ\n
From this moment begins our greedy search. We hypothesize hypotheses, select hypermaparmeters, train, and then check whether the hypothesis was correct using the resulting metrics. This stage is one of the most interesting. Offhand, I can say that the best results can be obtained on a pre-trained model with the parameters below (well + - still try to experiment)
\n
","metadata":{}},{"cell_type":"code","source":"def main():\n model = YOLO(\"yolov8x-seg.pt\")\n model.train(\n # Project\n project=\"HuBMAP\",\n name=\"yolov8x-seg\",\n\n # Random Seed parameters\n deterministic=True,\n seed=43,\n\n # Data & model parameters\n data=\"/kaggle/working/dataset/coco.yaml\", \n save=True,\n save_period=5,\n pretrained=True,\n imgsz=512,\n\n # Training parameters\n epochs=20,\n batch=4,\n workers=8,\n val=True,\n device=0,\n\n # Optimization parameters\n lr0=0.018,\n patience=3,\n optimizer=\"SGD\",\n momentum=0.947,\n weight_decay=0.0005,\n close_mosaic=3,\n )\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n Please follow the link. Sign up and then paste your API key into the box that pops up below. Do not disclose your key to anyone. Here it is required so that you can track the training\n
","metadata":{}},{"cell_type":"code","source":"if __name__ == '__main__':\n main()","metadata":{"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Predict\n\n
\nThe training has come to an end and now it is time to look at the results of the work of the neural network\n\n
","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nfrom PIL import Image","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n Choose any picture\n
","metadata":{}},{"cell_type":"code","source":"dirlist = os.listdir(\"/kaggle/working/dataset/val/images\")\nprint(dirlist[:5])","metadata":{"execution":{"iopub.status.busy":"2023-08-10T20:36:58.116039Z","iopub.execute_input":"2023-08-10T20:36:58.116753Z","iopub.status.idle":"2023-08-10T20:36:58.122286Z","shell.execute_reply.started":"2023-08-10T20:36:58.116717Z","shell.execute_reply":"2023-08-10T20:36:58.121342Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n Displaying the image\n
","metadata":{}},{"cell_type":"code","source":"# model = YOLO(\"/kaggle/working/HuBMAP/yolov8x-seg/weights/best.pt\")\n#model = YOLO(\"yolov8x-seg.pt\")\nmodel = YOLO('yolov8n-seg.pt')\nhistory = model.predict(\"../working/dataset/val/images/ed6a92a9410c.tif\")[0]\nimage = history.plot()\nplt.imshow(image)\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Let's build the graphs ","metadata":{}},{"cell_type":"markdown","source":"
\n Losses & Recall & Precision & mAP\n
","metadata":{}},{"cell_type":"code","source":"F1_curve = Image.open(\"/kaggle/working/HuBMAP/yolov8x-seg/results.png\")\nplt.figure(figsize=(15,20))\nplt.imshow(F1_curve)\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n Train Batch\n
","metadata":{}},{"cell_type":"code","source":"P_curve = Image.open(\"/kaggle/working/HuBMAP/yolov8x-seg/train_batch5561.jpg\")\nplt.imshow(P_curve)\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n What about submission? ๐Ÿ˜ธ\n
We have come a long way, done data analysis, trained the model, but what about the submission? We want to send the results, right? Well, I had difficulties with this, and therefore, in order to make life easier for myself, it is possible to help someone, I wrote several classes that allow you to quickly submit without delving into the difficulties that I encountered.
\n
","metadata":{}},{"cell_type":"markdown","source":"
\n Encode Binary Mask\n
","metadata":{}},{"cell_type":"code","source":"import base64\nimport numpy as np\nimport torch\nfrom pycocotools import _mask as coco_mask\nimport typing as t\nimport zlib\nimport pandas as pd\nimport torchvision.transforms as T\nfrom ultralytics import YOLO\nfrom PIL import Image","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class EncodeBinaryMask:\n @staticmethod\n def __checking_mask(mask: np.ndarray) -> np.ndarray:\n if mask.dtype != np.bool:\n raise ValueError(\n \"expects a binary mask, received dtype == %s\" %\n mask.dtype\n )\n return mask\n\n @staticmethod\n def __convert_mask(mask: np.ndarray):\n mask_to_encode = mask.astype(np.uint8)\n mask_to_encode = np.asfortranarray(mask_to_encode)\n return mask_to_encode\n\n @staticmethod\n def __compress_encode(encoded_mask) -> t.Text:\n binary_str = zlib.compress(encoded_mask, zlib.Z_BEST_COMPRESSION)\n base64_str = base64.b64encode(binary_str)\n return base64_str\n\n def __call__(self, mask: np.ndarray) -> t.Text:\n mask = self.__checking_mask(mask)\n mask_to_encode = self.__convert_mask(mask)\n encoded_mask = coco_mask.encode(mask_to_encode)[0][\"counts\"]\n base64_str = self.__compress_encode(encoded_mask)\n return base64_str","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n Submission\n
","metadata":{}},{"cell_type":"code","source":"class Submission:\n def __init__(self, dirpath: str, model: torch.nn.Module):\n self.__eval_transforms = self.get_transforms()\n self.__model = model\n self.__encoder = EncodeBinaryMask()\n self.__dirpath = dirpath\n self.__filenames = os.listdir(dirpath)\n self.height = 512\n self.width = 512\n \n self.__submission_dict = {\n \"id\": [],\n \"height\": [],\n \"width\": [],\n \"prediction_string\": []\n }\n \n self.submission = None\n \n @staticmethod\n def get_transforms():\n return T.Compose([\n T.ToTensor(),\n T.Resize(size=(512, 512)),\n T.Normalize(mean=[0.485, 0.456, 0.406],\n std=[0.229, 0.224, 0.225])\n ])\n\n def __len__(self):\n return len(self.__filenames)\n\n def __get_columns(self) -> None:\n for filename in self.__filenames:\n path = self.__get_image_path(filename)\n masks = self.__forward(path)\n identifier, height, width, prediction_string = self.__get_cells(filename, masks)\n self.__update_columns(identifier, height, width, prediction_string)\n\n def __update_columns(self, identifier: str, height: int, width: int, prediction_string: str) -> None:\n self.__submission_dict[\"id\"].append(identifier)\n self.__submission_dict[\"height\"].append(height)\n self.__submission_dict[\"width\"].append(width)\n self.__submission_dict[\"prediction_string\"].append(prediction_string)\n\n def __get_cells(self, filename: str, masks: list):\n prediction_string = \"\"\n prediction_string = self.__get_prediction_string(masks, prediction_string)\n identifier = filename.split(\".\")[0]\n return identifier, self.height, self.width, prediction_string\n\n def __get_prediction_string(self, masks: list, prediction_string: str) -> str:\n if masks:\n for outputs in masks:\n mask = outputs[\"mask\"]\n mask = np.where(mask > 0.5, 1, 0).astype(np.bool)\n base64_str = self.__encoder(mask)\n confidence = outputs[\"confidence\"]\n prediction_string += f\"0 {confidence} {base64_str.decode('utf-8')} \"\n else:\n return \"\"\n return prediction_string\n\n def __get_image_path(self, filename: str) -> str:\n return os.path.join(\n self.__dirpath, filename\n )\n\n def __get_image(self, path: str) -> torch.Tensor:\n image = Image.open(path)\n image = np.asarray(image)\n image = self.__eval_transforms(image)\n return image\n\n def __forward(self, image: torch.tensor) -> list:\n masks = self.__model(image) \n return masks \n\n def submit(self) -> None:\n if not self.submission:\n self.__get_columns()\n self.submission = pd.DataFrame(self.__submission_dict)\n self.submission = self.submission.set_index('id')\n self.submission.to_csv(\"submission.csv\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n Wrapper class for YOLO model\n
","metadata":{}},{"cell_type":"code","source":"class BestYolo:\n def __init__(self, conf: float = 0.05):\n self.model_path = \"/kaggle/working/HuBMAP/yolov8x-seg/weights/best.pt\"\n self.model = self.get_model()\n self.conf = conf\n \n def get_model(self) -> YOLO:\n return YOLO(self.model_path)\n \n def __call__(self, source) -> list[dict, ...]:\n sublist = []\n result = self.model(source)[0]\n if result.masks:\n for i in range(len(result.masks.data)):\n conf = round(float(result.boxes.conf[i]), 2)\n mask = np.expand_dims(result.masks.data[i].cpu().numpy(), axis=0).transpose(1,2,0)\n \n if int(result.boxes.cls[i]) == 0 and conf >= self.conf:\n sublist.append({\"mask\": mask, \"confidence\": conf})\n else:\n continue\n return sublist\n else:\n return None","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"__TEST_PATH = \"/kaggle/input/hubmap-hacking-the-human-vasculature/test\"\nmodel = BestYolo()\nsub = Submission(dirpath=__TEST_PATH, model=model)\nsub.submit()","metadata":{"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub.submission.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n Thanks a lot for making it to the end\n
I hope you have found this work useful. I will be very grateful if you vote for this work, if it really helped you. Good luck ๐Ÿ™ƒ
\n
","metadata":{}},{"cell_type":"markdown","source":"![](https://media1.giphy.com/media/cCaSeXFNKlu6zBtSGd/giphy.gif)","metadata":{}}]}