{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# <B><u>Whale&Dolphin Background Removed Images (1/4)</u></B>","metadata":{}},{"cell_type":"markdown","source":"# <B>[1] Introduction</B>","metadata":{}},{"cell_type":"markdown","source":"There is no annotation data, which shows position of whale and dolphin, in \"Happywhale - Whale and Dolphin Identification\" competition dataset. It is difficult to create annotation data in a kernel because of limitations of kaggle notebook. So it is created in the series of the following kernels,\n- [\"Whale&Dolphin Background Removed Images (1/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-1-4) (This kernel),\n- [\"Whale&Dolphin Background Removed Images (2/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-2-4),\n- [\"Whale&Dolphin Background Removed Images (3/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-3-4),\n- [\"Whale&Dolphin Background Removed Images (4/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-4-4),\n- [\"Annotation Data for Detecting Whale&Dolphin\"](https://www.kaggle.com/code/acchiko/annotation-data-for-detecting-whale-dolphin),\n- [\"Whale&Dolphin Cropped Images (1/3)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-cropped-images-1-3),\n- [\"Whale&Dolphin Cropped Images (2/3)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-cropped-images-2-3),\n- [\"Whale&Dolphin Cropped Images (3/3)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-cropped-images-3-3).\n\nIf the kernels are loaded, created annotation data and cropped images can be available from the following path for example,\n- Annotation data : /kaggle/input/annotation-data-for-detecting-whale-dolphin/train_with_annotation.csv,\n- Background removed images : /kaggle/input/annotation-data-for-detecting-whale-dolphin/nobg/train_image/*.png,\n- Cropped images : /kaggle/input/whale-dolphin-cropped-images-3-3/cropped/train_image/*.png.","metadata":{}},{"cell_type":"markdown","source":"NOTE1 : The author is a beginner of Kaggle/MachineLearning/Python/English. So the kernel may have several bugs/wrongs. I am happy to get your comments. Thank you in advance for your kind advice to make the kernel so NICE! and to make me NICE deep learning guy!! ","metadata":{}},{"cell_type":"markdown","source":"NOTE2 : Utility scripts for visualization of dataset for \"Happywhale - Whale and Dolphin Identification\" competition is defined in my other kernel [\"Utility Functions for Visualization of Dataset\"](https://www.kaggle.com/code/acchiko/utility-functions-for-visualization-of-dataset). The way to create/use utility scripts is summarized in my other kernel [\"How to Create Utility Scripts\"](https://www.kaggle.com/code/acchiko/utility-functions-for-visualization-of-dataset).","metadata":{}},{"cell_type":"markdown","source":"NOTE3 : Dataset for \"Happywhale - Whale and Dolphin Identification\" competition is visualized with [Plotly](https://plotly.com/python/) and [Matplotlib](https://matplotlib.org/) in my other kernel [\"Preview of Whale&Dolphin Dataset with Plotly/Matplotlib\"](https://www.kaggle.com/acchiko/preview-of-whale-dolphin-dataset-with-plotly-matpl). It may help us to get some insight into strategy of training, data augumentation, etc.","metadata":{}},{"cell_type":"markdown","source":"# <B>[2] Preparation of dataset</B>","metadata":{}},{"cell_type":"markdown","source":"## [2-1] Loading dataset","metadata":{}},{"cell_type":"markdown","source":"Dataset for \"Happywhale - Whale and Dolphin Identification\" competition is loaded by clicking the following items in the sidebar of kaggle notebook,","metadata":{}},{"cell_type":"markdown","source":"###  \"+ Add data\" -> \"Competition Data\" -> \"Add (Happywhale - Whale and Dolphin Identification)\".","metadata":{}},{"cell_type":"markdown","source":"If it succeeds, the dataset are loaded to the following path.","metadata":{}},{"cell_type":"code","source":"path_to_dir_happywhale_data = \"/kaggle/input/happy-whale-and-dolphin\"\n!ls {path_to_dir_happywhale_data}","metadata":{"execution":{"iopub.status.busy":"2022-04-02T09:28:48.669401Z","iopub.execute_input":"2022-04-02T09:28:48.669696Z","iopub.status.idle":"2022-04-02T09:28:48.975169Z","shell.execute_reply.started":"2022-04-02T09:28:48.669665Z","shell.execute_reply":"2022-04-02T09:28:48.974142Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [2-2] Showing contents of dataset ","metadata":{}},{"cell_type":"markdown","source":"Contents of metadata for train images is shown.","metadata":{}},{"cell_type":"code","source":"path_to_happywhale_train_metadata = \"%s/train.csv\" % path_to_dir_happywhale_data\n!head -5 {path_to_happywhale_train_metadata}","metadata":{"execution":{"iopub.status.busy":"2022-04-02T09:28:48.977015Z","iopub.execute_input":"2022-04-02T09:28:48.977409Z","iopub.status.idle":"2022-04-02T09:28:49.274682Z","shell.execute_reply.started":"2022-04-02T09:28:48.977378Z","shell.execute_reply":"2022-04-02T09:28:49.273839Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"List of train/test images are shown.","metadata":{}},{"cell_type":"code","source":"path_to_dir_happywhale_train_images = \"%s/train_images\" % path_to_dir_happywhale_data\n!ls {path_to_dir_happywhale_train_images} | head -5","metadata":{"execution":{"iopub.status.busy":"2022-04-02T09:28:49.276203Z","iopub.execute_input":"2022-04-02T09:28:49.276452Z","iopub.status.idle":"2022-04-02T09:28:49.701040Z","shell.execute_reply.started":"2022-04-02T09:28:49.276421Z","shell.execute_reply":"2022-04-02T09:28:49.700409Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"path_to_dir_happywhale_test_images = \"%s/test_images\" % path_to_dir_happywhale_data\n!ls {path_to_dir_happywhale_test_images} | head -5","metadata":{"execution":{"iopub.status.busy":"2022-04-02T09:28:49.702236Z","iopub.execute_input":"2022-04-02T09:28:49.703003Z","iopub.status.idle":"2022-04-02T09:28:50.045389Z","shell.execute_reply.started":"2022-04-02T09:28:49.702963Z","shell.execute_reply":"2022-04-02T09:28:50.044691Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [2-3] Creation of metadata for test images","metadata":{}},{"cell_type":"markdown","source":"The metadata for train images exists, but the one for test images is not exists. So the dummy metadata for test images is created.","metadata":{}},{"cell_type":"code","source":"# Creates metadata for test images. \npath_to_happywhale_test_metadata = \"/kaggle/working/test.csv\"\n\n!echo \"image,species,individual_id\" > {path_to_happywhale_test_metadata}\n!ls {path_to_dir_happywhale_test_images} | sed \"s/.jpg/.jpg,unknown,unknown/g\" >> {path_to_happywhale_test_metadata}\n\n# Shows contents of created metadata.\n!head -5 {path_to_happywhale_test_metadata}","metadata":{"execution":{"iopub.status.busy":"2022-04-02T09:28:50.047379Z","iopub.execute_input":"2022-04-02T09:28:50.047591Z","iopub.status.idle":"2022-04-02T09:28:51.009859Z","shell.execute_reply.started":"2022-04-02T09:28:50.047564Z","shell.execute_reply":"2022-04-02T09:28:51.008700Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [2-4] Creation of working directories","metadata":{}},{"cell_type":"markdown","source":"Working directories for saving processed images(background removed/cropped images) are created.","metadata":{}},{"cell_type":"code","source":"# Creates working directory for saving background removed images.\npath_to_dir_nobg_train_images = \"/kaggle/working/nobg/train_images\"\n!mkdir -p {path_to_dir_nobg_train_images}","metadata":{"execution":{"iopub.status.busy":"2022-04-02T09:28:51.011791Z","iopub.execute_input":"2022-04-02T09:28:51.012030Z","iopub.status.idle":"2022-04-02T09:28:51.307880Z","shell.execute_reply.started":"2022-04-02T09:28:51.012005Z","shell.execute_reply":"2022-04-02T09:28:51.306503Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Creates working directory for saving cropped images.\npath_to_dir_cropped_train_images = \"/kaggle/working/cropped/train_images\"\n!mkdir -p {path_to_dir_cropped_train_images}","metadata":{"execution":{"iopub.status.busy":"2022-04-02T09:28:51.309414Z","iopub.execute_input":"2022-04-02T09:28:51.309652Z","iopub.status.idle":"2022-04-02T09:28:51.609396Z","shell.execute_reply.started":"2022-04-02T09:28:51.309615Z","shell.execute_reply":"2022-04-02T09:28:51.608092Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Shows created directories.\n!ls /kaggle/working/*","metadata":{"execution":{"iopub.status.busy":"2022-04-02T09:28:51.611461Z","iopub.execute_input":"2022-04-02T09:28:51.611752Z","iopub.status.idle":"2022-04-02T09:28:51.906383Z","shell.execute_reply.started":"2022-04-02T09:28:51.611726Z","shell.execute_reply":"2022-04-02T09:28:51.905487Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [2-5] Loading utility scripts","metadata":{}},{"cell_type":"markdown","source":"Utility scripts for visualization of \"Happywhale - Whale and Dolphin Identification\" competition dataset, which is defined in the other kernel [\"Utility Functions for Visualization of Dataset\"](https://www.kaggle.com/code/acchiko/utility-functions-for-visualization-of-dataset), are loaded. The way to use/create utility scripts is summarized in the other kernel [\"How to Create Utility Scripts\"](https://www.kaggle.com/code/acchiko/how-to-use-create-utility-scripts).","metadata":{}},{"cell_type":"code","source":"import utility_functions_for_visualization_of_dataset as myutils","metadata":{"execution":{"iopub.status.busy":"2022-04-02T09:28:51.907750Z","iopub.execute_input":"2022-04-02T09:28:51.908331Z","iopub.status.idle":"2022-04-02T09:28:51.914216Z","shell.execute_reply.started":"2022-04-02T09:28:51.908278Z","shell.execute_reply":"2022-04-02T09:28:51.913079Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [2-6] Showing images","metadata":{}},{"cell_type":"markdown","source":"Train/Test images are shown. First, class for processing train/test images and metadata is defined.","metadata":{}},{"cell_type":"code","source":"# Imports required libs.\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom PIL import Image, ImageDraw\nimport plotly.graph_objects as go\nfrom plotly.subplots import make_subplots\nimport os\nfrom tqdm import tqdm","metadata":{"execution":{"iopub.status.busy":"2022-04-02T09:28:51.915628Z","iopub.execute_input":"2022-04-02T09:28:51.915837Z","iopub.status.idle":"2022-04-02T09:28:51.931338Z","shell.execute_reply.started":"2022-04-02T09:28:51.915809Z","shell.execute_reply":"2022-04-02T09:28:51.930244Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Defines class for processing train/test images and metadata.\nclass WhaleAndDolphin():\n    def __init__(self, path_to_metadata, path_to_dir_images, \\\n                 path_to_dir_nobg_images, path_to_dir_cropped_images):\n        self._path_to_metadata = path_to_metadata\n        self._path_to_dir_images = path_to_dir_images\n        self._path_to_dir_nobg_images = path_to_dir_nobg_images\n        self._path_to_dir_cropped_images = path_to_dir_cropped_images\n        \n        # Loads metadata to variable \"_metadata_all\"\n        self._metadata_all = pd.read_csv(path_to_metadata)\n        \n        # Adds several colmuns.\n        path_to_images = \\\n            [\"%s/%s\" % (path_to_dir_images, row.image) \\\n             for row in self._metadata_all.itertuples()]\n        self._metadata_all[\"path_to_image\"] = path_to_images\n        \n        path_to_nobg_images = \\\n            [\"%s/%s\" % (path_to_dir_nobg_images, row.image.replace(\".jpg\", \".png\")) \\\n             for row in self._metadata_all.itertuples()]\n        self._metadata_all[\"path_to_nobg_image\"] = path_to_nobg_images\n        \n        path_to_cropped_images = \\\n            [\"%s/%s\" % (path_to_dir_cropped_images, row.image.replace(\".jpg\", \".png\")) \\\n             for row in self._metadata_all.itertuples()]\n        self._metadata_all[\"path_to_cropped_image\"] = path_to_cropped_images\n        \n        annotations_xyxy = \\\n            [[] for row in self._metadata_all.itertuples()]\n        self._metadata_all[\"annotations_xyxy\"] = annotations_xyxy\n        \n        # Copies the metadata for processing it.\n        self._metadata = self._metadata_all.copy()\n        \n        self._all_species = self.getSpecies()\n        self._all_individual_ids = self.getIndividualIDs()\n        \n    def resetMetadata(self, initialize=False):\n        if hasattr(self, \"_metadata_tmp\") and not initialize:\n            self._metadata = self._metadata_tmp.copy()\n        else:\n            self._metadata = self._metadata_all.copy()\n            \n    def saveMetadataTemporary(self):\n        self._metadata_tmp = self._metadata.copy()\n        \n    def filterMetadata(self, query=\"index > -1\"):\n        sliced_metadata = \\\n            self._metadata.query(query).reset_index(drop=True)\n        self._metadata = sliced_metadata.copy()\n        \n    def filterMetadataNoBackgroundImageExistence(self):\n        indices = []\n        for row in self._metadata.itertuples():\n            if not os.path.exists(row.path_to_nobg_image):\n                indices.append(row.Index)\n                \n        sliced_metadata = self._metadata.drop(index=indices).reset_index(drop=True)\n        self._metadata = sliced_metadata.copy()\n        \n    def writeMetadata(self, path_to_metadata):\n        self._metadata.to_csv(path_to_metadata, index=False)\n        \n    def getMetadata(self):\n        return self._metadata\n    \n    def getSpecies(self):\n        return self._metadata[\"species\"].unique()\n    \n    def _species2id(self, species):\n        return np.where(self._all_species == species)\n    \n    def getIndividualIDs(self):\n        return self._metadata[\"individual_id\"].unique()\n    \n    def showImagesTile(self, num_cols=4, draw_annotations=False):\n        metadata = self._metadata\n        titles = [row.image for row in metadata.itertuples()]\n        path_to_images = [row.path_to_image \\\n                          for row in metadata.itertuples()]\n        images = myutils.getImages(path_to_images)\n        if \"annotations_xyxy\" in metadata.columns and draw_annotations:\n            annotations_xyxy_for_images = [row.annotations_xyxy \\\n                                           for row in metadata.itertuples()]\n            texts_for_images = [[\"\" for _ in \\\n                                 range(len(row.annotations_xyxy))] \\\n                                 for row in metadata.itertuples()]\n            myutils.drawAnnotations( \\\n                images, \\\n                annotations_xyxy_for_images=annotations_xyxy_for_images, \\\n                texts_for_images=texts_for_images, \\\n                line_color=\"green\", line_width=3, text_color=\"green\" \\\n            )\n        myutils.showImagesTile(titles, images, num_cols=num_cols)\n        \n    def showProcessedImagesTile(self, num_cols=3, draw_annotations=False):\n        metadata = self._metadata\n        titles, path_to_images = [], []\n        for row in metadata.itertuples():\n            titles.append(\"%s (Org.)\" % row.image)\n            path_to_images.append(row.path_to_image)\n            \n            titles.append(\"%s (BG. removed)\" % row.image)\n            path_to_images.append(row.path_to_nobg_image)\n            \n            titles.append(\"%s (Cropped)\" % row.image)\n            path_to_images.append(row.path_to_cropped_image)\n        \n        images = myutils.getImages(path_to_images)\n        if \"annotations_xyxy\" in metadata.columns and draw_annotations:\n            annotations_xyxy_for_images = [row.annotations_xyxy \\\n                                           for row in metadata.itertuples()]\n            texts_for_images = [[\"\" for _ in \\\n                                 range(len(row.annotations_xyxy))] \\\n                                 for row in metadata.itertuples()]\n            myutils.drawAnnotations( \\\n                images[::3], \\\n                annotations_xyxy_for_images=annotations_xyxy_for_images, \\\n                texts_for_images=texts_for_images, \\\n                line_color=\"red\", line_width=3, text_color=\"red\" \\\n            ) # For only org. images.\n        myutils.showImagesTile(titles, images, num_cols=num_cols)\n        \n    def showIndividualImagesTile(self, num_cols=4, \\\n                                 max_num_individual_images=4, \\\n                                 max_num_individuals=10, \\\n                                 draw_annotations=False):\n        self.saveMetadataTemporary()\n        \n        individual_ids = self.getIndividualIDs()\n        for individual_id in individual_ids[:max_num_individuals]:\n            print()\n            print(\"Individual ID : %s\" % individual_id)\n            self.filterMetadata(query=\"individual_id == \\\"%s\\\"\" % individual_id)\n            self.filterMetadata(query=\"index < %d\" % max_num_individual_images)\n            self.showImagesTile( \\\n                num_cols=num_cols, \\\n                draw_annotations=draw_annotations\n            )\n            self.resetMetadata()\n            \n    def removeBackground(self):\n        metadata = self._metadata\n        path_to_inputs = [row.path_to_image \\\n                          for row in metadata.itertuples()]\n        path_to_outputs = [row.path_to_nobg_image \\\n                           for row in metadata.itertuples()]\n        \n        for path_to_input, path_to_output in \\\n            zip(path_to_inputs, path_to_outputs):\n            !backgroundremover -i {path_to_input} -o {path_to_output}\n        \n    def calculateAnnotationsXyxy(self):\n        batch_size = 100\n        num_batches = len(self._metadata) // batch_size + 1\n        \n        for i_batch in range(num_batches):\n            i_start = i_batch * batch_size\n            i_end = i_start + batch_size\n            metadata = self._metadata.iloc[i_start:i_end]\n            \n            path_to_nobg_images = [row.path_to_nobg_image \\\n                                   for row in metadata.itertuples()]\n            nobg_images = myutils.getImages(path_to_nobg_images)\n            fixed_images = [Image.eval(nobg_image, self._removeBugPixel) \\\n                            for nobg_image in nobg_images]\n            class_ids = [self._species2id(row.species) for row \\\n                         in metadata.itertuples()]\n            \n            annotations_xyxy = []\n            for nobg_image, class_id in zip(fixed_images, class_ids):\n                x_min, y_min, x_max, y_max = nobg_image.getbbox() # Bounding box of non-zero region\n                confidence = 1.0 # Dummy\n                annotation_xyxy = myutils._annotationXyxy(class_id, \\\n                                                          x_min, y_min, \\\n                                                          x_max, y_max, \\\n                                                          confidence)\n                annotations_xyxy.append([annotation_xyxy])\n                \n            self._metadata[\"annotations_xyxy\"].iloc[i_start:i_end] = \\\n                annotations_xyxy\n    \n    def _removeBugPixel(self, pixel_value):\n        if pixel_value == 1:\n            return 0\n        else:\n            return pixel_value\n        \n    def cropObject(self):\n        batch_size = 100\n        num_batches = len(self._metadata) // batch_size + 1\n        \n        for i_batch in range(num_batches):\n            i_start = i_batch * batch_size\n            i_end = i_start + batch_size\n            metadata = self._metadata.iloc[i_start:i_end]\n            \n            path_to_inputs = [row.path_to_image \\\n                              for row in metadata.itertuples()]\n            path_to_outputs = [row.path_to_cropped_image \\\n                               for row in metadata.itertuples()]\n            annotations_xyxy = [row.annotations_xyxy for row \\\n                                in metadata.itertuples()]\n            \n            for path_to_input, path_to_output, annotations_xyxy in \\\n                zip(path_to_inputs, path_to_outputs, annotations_xyxy):\n                \n                image = Image.open(path_to_input)\n                annotation_xyxy = \\\n                    self._maxConfidenceAnnotation(annotations_xyxy)\n                x_min = annotation_xyxy[\"x_min\"]\n                y_min = annotation_xyxy[\"y_min\"]\n                x_max = annotation_xyxy[\"x_max\"]\n                y_max = annotation_xyxy[\"y_max\"]\n                image_cropped = image.crop((x_min, y_min, x_max, y_max))\n                image_cropped.save(path_to_output)\n            \n    def _maxConfidenceAnnotation(self, annotations_xyxy):\n        confidences = np.array([annotation_xyxy[\"confidence\"] \\\n                                for annotation_xyxy in annotations_xyxy])\n        index = np.argmax(confidences)\n        return annotations_xyxy[index]","metadata":{"execution":{"iopub.status.busy":"2022-04-02T09:28:51.933101Z","iopub.execute_input":"2022-04-02T09:28:51.933800Z","iopub.status.idle":"2022-04-02T09:28:52.026164Z","shell.execute_reply.started":"2022-04-02T09:28:51.933722Z","shell.execute_reply":"2022-04-02T09:28:52.025335Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Some of train images are shown as example. Test images can be shown using the same class.","metadata":{}},{"cell_type":"code","source":"# Loads metadata for train images.\nwhale_and_dolphin = WhaleAndDolphin(\n    path_to_metadata=path_to_happywhale_train_metadata,\n    path_to_dir_images=path_to_dir_happywhale_train_images,\n    path_to_dir_nobg_images=path_to_dir_nobg_train_images,\n    path_to_dir_cropped_images=path_to_dir_cropped_train_images\n)","metadata":{"execution":{"iopub.status.busy":"2022-04-02T09:28:52.027519Z","iopub.execute_input":"2022-04-02T09:28:52.027759Z","iopub.status.idle":"2022-04-02T09:28:52.621761Z","shell.execute_reply.started":"2022-04-02T09:28:52.027721Z","shell.execute_reply":"2022-04-02T09:28:52.620891Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Shows train images for the first 3 individuals for each species.\nnum_cols = 4\nmax_num_individual_images = 4\nmax_num_individuals = 3\n\nall_species = whale_and_dolphin.getSpecies()\nfor i, species in enumerate(all_species):\n    whale_and_dolphin.filterMetadata(query=\"species == \\\"%s\\\"\" % species)\n    \n    print()\n    print(\"--------------------------------------------------\")\n    print()\n    print(\"   Images for species No.%02d %s\" % (i, species))\n    print()\n    print(\"--------------------------------------------------\")\n    whale_and_dolphin.showIndividualImagesTile( \\\n        num_cols=num_cols, \\\n        max_num_individual_images=max_num_individual_images, \\\n        max_num_individuals=max_num_individuals \\\n    )\n    print()\n    print()\n    \n    whale_and_dolphin.resetMetadata(initialize=True)","metadata":{"execution":{"iopub.status.busy":"2022-04-02T09:28:52.623361Z","iopub.execute_input":"2022-04-02T09:28:52.623809Z","iopub.status.idle":"2022-04-02T09:31:56.222442Z","shell.execute_reply.started":"2022-04-02T09:28:52.623759Z","shell.execute_reply":"2022-04-02T09:31:56.220937Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <B>[3] Creation of annotation data</B>","metadata":{}},{"cell_type":"markdown","source":"Annotation data is created with the following steps,\n- Removing background of image,\n- Calculating bounding box,\n- Cropping bounding box,\n- Showing processed images.","metadata":{}},{"cell_type":"markdown","source":"## [3-1] Removing background of image","metadata":{}},{"cell_type":"markdown","source":"Background of image is removed with the command line tool [\"backgroundremover\"](https://github.com/nadermx/backgroundremover) to make easy to calculate bounding box. Sometimes background is not removed properly with the tool, but it works well for almost cases.","metadata":{}},{"cell_type":"markdown","source":"It is difficult to remove background with the tool in a kernel because of RAM utilization limitation of kaggle notebook (~4000 images can be processed in a kernel within the limitation.). So it is done in the series of the following kernels,\n- [\"Whale&Dolphin Background Removed Images (1/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-1-4) (This kernel),\n- [\"Whale&Dolphin Background Removed Images (2/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-2-4),\n- [\"Whale&Dolphin Background Removed Images (3/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-3-4),\n- [\"Whale&Dolphin Background Removed Images (4/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-4-4).","metadata":{}},{"cell_type":"markdown","source":"First, the tool is installed.","metadata":{}},{"cell_type":"code","source":"!pip install backgroundremover","metadata":{"execution":{"iopub.status.busy":"2022-04-02T09:31:56.228036Z","iopub.execute_input":"2022-04-02T09:31:56.228891Z","iopub.status.idle":"2022-04-02T09:33:15.504345Z","shell.execute_reply.started":"2022-04-02T09:31:56.228756Z","shell.execute_reply":"2022-04-02T09:33:15.503249Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Then background is removed.","metadata":{}},{"cell_type":"code","source":"# Loads metadata for train images.\nwhale_and_dolphin = WhaleAndDolphin(\n    path_to_metadata=path_to_happywhale_train_metadata,\n    path_to_dir_images=path_to_dir_happywhale_train_images,\n    path_to_dir_nobg_images=path_to_dir_nobg_train_images,\n    path_to_dir_cropped_images=path_to_dir_cropped_train_images\n)","metadata":{"execution":{"iopub.status.busy":"2022-04-02T09:33:15.506019Z","iopub.execute_input":"2022-04-02T09:33:15.506268Z","iopub.status.idle":"2022-04-02T09:33:16.003640Z","shell.execute_reply.started":"2022-04-02T09:33:15.506243Z","shell.execute_reply":"2022-04-02T09:33:16.002768Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Removes background. Limits number of processing images because of kaggle notebook limitation. Sets same number for each species as possible.\nnum_images_per_species = int(4000 / len(all_species))\ni_start = 0 * num_images_per_species\ni_end = i_start + num_images_per_species\n\nall_species = whale_and_dolphin.getSpecies()\nwhale_and_dolphin.saveMetadataTemporary()\n\nfor i, species in enumerate(all_species):\n    whale_and_dolphin.filterMetadata(query=\"species == \\\"%s\\\"\" % species)\n    whale_and_dolphin.filterMetadata(query=\"%d <= index < %d\" % (i_start, i_end))\n    print()\n    print(\"--------------------------------------------------\")\n    print()\n    print(\"   Processing images for species No.%02d %s\" % (i, species))\n    print()\n    print(\"--------------------------------------------------\")\n    whale_and_dolphin.removeBackground()\n    print()\n    \n    whale_and_dolphin.resetMetadata()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If it succeeds, background removed images are saved in the following directory.","metadata":{}},{"cell_type":"code","source":"!ls {path_to_dir_nobg_train_images} | cat -n | tail -5","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}