{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# <B><u>Whale&Dolphin Cropped Images (1/3)</u></B>","metadata":{}},{"cell_type":"markdown","source":"# <B>[1] Introduction</B>","metadata":{}},{"cell_type":"markdown","source":"There is no annotation data, which shows position of whale and dolphin, in \"Happywhale - Whale and Dolphin Identification\" competition dataset. It is difficult to create annotation data in a kernel because of limitations of kaggle notebook. So it is created in the series of the following kernels,\n- [\"Whale&Dolphin Background Removed Images (1/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-1-4),\n- [\"Whale&Dolphin Background Removed Images (2/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-2-4),\n- [\"Whale&Dolphin Background Removed Images (3/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-3-4),\n- [\"Whale&Dolphin Background Removed Images (4/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-4-4),\n- [\"Annotation Data for Detecting Whale&Dolphin\"](https://www.kaggle.com/code/acchiko/annotation-data-for-detecting-whale-dolphin),\n- [\"Whale&Dolphin Cropped Images (1/3)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-cropped-images-1-3) (This kernel),\n- [\"Whale&Dolphin Cropped Images (2/3)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-cropped-images-2-3),\n- [\"Whale&Dolphin Cropped Images (3/3)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-cropped-images-3-3).\n\nIf the kernels are loaded, created annotation data and cropped images can be available from the following path for example,\n- Annotation data : /kaggle/input/annotation-data-for-detecting-whale-dolphin/train_with_annotation.csv,\n- Background removed images : /kaggle/input/annotation-data-for-detecting-whale-dolphin/nobg/train_image/*.png,\n- Cropped images : /kaggle/input/whale-dolphin-cropped-images-3-3/cropped/train_image/*.png.","metadata":{}},{"cell_type":"markdown","source":"NOTE1 : The author is a beginner of Kaggle/MachineLearning/Python/English. So the kernel may have several bugs/wrongs. I am happy to get your comments. Thank you in advance for your kind advice to make the kernel so NICE! and to make me NICE deep learning guy!! ","metadata":{}},{"cell_type":"markdown","source":"NOTE2 : Utility scripts for visualization of dataset for \"Happywhale - Whale and Dolphin Identification\" competition is defined in my other kernel [\"Utility Functions for Visualization of Dataset\"](https://www.kaggle.com/code/acchiko/utility-functions-for-visualization-of-dataset). The way to create/use utility scripts is summarized in my other kernel [\"How to Create Utility Scripts\"](https://www.kaggle.com/code/acchiko/utility-functions-for-visualization-of-dataset).","metadata":{}},{"cell_type":"markdown","source":"NOTE3 : Dataset for \"Happywhale - Whale and Dolphin Identification\" competition is visualized with [Plotly](https://plotly.com/python/) and [Matplotlib](https://matplotlib.org/) in my other kernel [\"Preview of Whale&Dolphin Dataset with Plotly/Matplotlib\"](https://www.kaggle.com/acchiko/preview-of-whale-dolphin-dataset-with-plotly-matpl). It may help us to get some insight into strategy of training, data augumentation, etc.","metadata":{}},{"cell_type":"markdown","source":"# <B>[2] Preparation of dataset</B>","metadata":{}},{"cell_type":"markdown","source":"## [2-1] Loading dataset","metadata":{}},{"cell_type":"markdown","source":"Dataset for \"Happywhale - Whale and Dolphin Identification\" competition is loaded by clicking the following items in the sidebar of kaggle notebook,","metadata":{}},{"cell_type":"markdown","source":"###  \"+ Add data\" -> \"Competition Data\" -> \"Add (Happywhale - Whale and Dolphin Identification)\".","metadata":{}},{"cell_type":"markdown","source":"If it succeeds, the dataset are loaded to the following path.","metadata":{}},{"cell_type":"code","source":"path_to_dir_happywhale_data = \"/kaggle/input/happy-whale-and-dolphin\"\n!ls {path_to_dir_happywhale_data}","metadata":{"execution":{"iopub.status.busy":"2022-04-05T07:34:37.693129Z","iopub.execute_input":"2022-04-05T07:34:37.693460Z","iopub.status.idle":"2022-04-05T07:34:38.479579Z","shell.execute_reply.started":"2022-04-05T07:34:37.693376Z","shell.execute_reply":"2022-04-05T07:34:38.478447Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [2-2] Showing contents of dataset ","metadata":{}},{"cell_type":"markdown","source":"Contents of metadata for train images is shown.","metadata":{}},{"cell_type":"code","source":"path_to_happywhale_train_metadata = \"%s/train.csv\" % path_to_dir_happywhale_data\n!head -5 {path_to_happywhale_train_metadata}","metadata":{"execution":{"iopub.status.busy":"2022-04-05T07:34:38.483932Z","iopub.execute_input":"2022-04-05T07:34:38.484223Z","iopub.status.idle":"2022-04-05T07:34:39.257374Z","shell.execute_reply.started":"2022-04-05T07:34:38.484187Z","shell.execute_reply":"2022-04-05T07:34:39.256578Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"List of train/test images are shown.","metadata":{}},{"cell_type":"code","source":"path_to_dir_happywhale_train_images = \"%s/train_images\" % path_to_dir_happywhale_data\n!ls {path_to_dir_happywhale_train_images} | head -5","metadata":{"execution":{"iopub.status.busy":"2022-04-05T07:34:39.259037Z","iopub.execute_input":"2022-04-05T07:34:39.259713Z","iopub.status.idle":"2022-04-05T07:34:40.984599Z","shell.execute_reply.started":"2022-04-05T07:34:39.259660Z","shell.execute_reply":"2022-04-05T07:34:40.983721Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"path_to_dir_happywhale_test_images = \"%s/test_images\" % path_to_dir_happywhale_data\n!ls {path_to_dir_happywhale_test_images} | head -5","metadata":{"execution":{"iopub.status.busy":"2022-04-05T07:34:40.987318Z","iopub.execute_input":"2022-04-05T07:34:40.987607Z","iopub.status.idle":"2022-04-05T07:34:42.363676Z","shell.execute_reply.started":"2022-04-05T07:34:40.987573Z","shell.execute_reply":"2022-04-05T07:34:42.362733Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [2-3] Creation of metadata for test images","metadata":{}},{"cell_type":"markdown","source":"The metadata for train images exists, but the one for test images is not exists. So the dummy metadata for test images is created.","metadata":{}},{"cell_type":"code","source":"# Creates metadata for test images. \npath_to_happywhale_test_metadata = \"/kaggle/working/test.csv\"\n\n!echo \"image,species,individual_id\" > {path_to_happywhale_test_metadata}\n!ls {path_to_dir_happywhale_test_images} | sed \"s/.jpg/.jpg,unknown,unknown/g\" >> {path_to_happywhale_test_metadata}\n\n# Shows contents of created metadata.\n!head -5 {path_to_happywhale_test_metadata}","metadata":{"execution":{"iopub.status.busy":"2022-04-05T07:34:42.365772Z","iopub.execute_input":"2022-04-05T07:34:42.366172Z","iopub.status.idle":"2022-04-05T07:34:44.710053Z","shell.execute_reply.started":"2022-04-05T07:34:42.366129Z","shell.execute_reply":"2022-04-05T07:34:44.709289Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [2-4] Creation of working directories","metadata":{}},{"cell_type":"markdown","source":"Working directories for saving processed images(background removed/cropped images) are created.","metadata":{}},{"cell_type":"code","source":"# Creates working directory for saving background removed images.\npath_to_dir_nobg_train_images = \"/kaggle/working/nobg/train_images\"\n!mkdir -p {path_to_dir_nobg_train_images}","metadata":{"execution":{"iopub.status.busy":"2022-04-05T07:34:44.711591Z","iopub.execute_input":"2022-04-05T07:34:44.712205Z","iopub.status.idle":"2022-04-05T07:34:45.465678Z","shell.execute_reply.started":"2022-04-05T07:34:44.712166Z","shell.execute_reply":"2022-04-05T07:34:45.464701Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Creates working directory for saving cropped images.\npath_to_dir_cropped_train_images = \"/kaggle/working/cropped/train_images\"\n!mkdir -p {path_to_dir_cropped_train_images}","metadata":{"execution":{"iopub.status.busy":"2022-04-05T07:34:45.467556Z","iopub.execute_input":"2022-04-05T07:34:45.468197Z","iopub.status.idle":"2022-04-05T07:34:46.213458Z","shell.execute_reply.started":"2022-04-05T07:34:45.468146Z","shell.execute_reply":"2022-04-05T07:34:46.212566Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Shows created directories.\n!ls /kaggle/working/*","metadata":{"execution":{"iopub.status.busy":"2022-04-05T07:34:46.214953Z","iopub.execute_input":"2022-04-05T07:34:46.215771Z","iopub.status.idle":"2022-04-05T07:34:46.964482Z","shell.execute_reply.started":"2022-04-05T07:34:46.215734Z","shell.execute_reply":"2022-04-05T07:34:46.963445Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [2-5] Loading utility scripts","metadata":{}},{"cell_type":"markdown","source":"Utility scripts for visualization of \"Happywhale - Whale and Dolphin Identification\" competition dataset, which is defined in the other kernel [\"Utility Functions for Visualization of Dataset\"](https://www.kaggle.com/code/acchiko/utility-functions-for-visualization-of-dataset), are loaded. The way to use/create utility scripts is summarized in the other kernel [\"How to Create Utility Scripts\"](https://www.kaggle.com/code/acchiko/how-to-use-create-utility-scripts).","metadata":{}},{"cell_type":"code","source":"import utility_functions_for_visualization_of_dataset as myutils","metadata":{"execution":{"iopub.status.busy":"2022-04-05T07:34:46.966621Z","iopub.execute_input":"2022-04-05T07:34:46.966904Z","iopub.status.idle":"2022-04-05T07:34:46.999114Z","shell.execute_reply.started":"2022-04-05T07:34:46.966870Z","shell.execute_reply":"2022-04-05T07:34:46.998378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [2-6] Showing images","metadata":{}},{"cell_type":"markdown","source":"Train/Test images are shown. First, class for processing train/test images and metadata is defined.","metadata":{}},{"cell_type":"code","source":"# Imports required libs.\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom PIL import Image, ImageDraw\nimport plotly.graph_objects as go\nfrom plotly.subplots import make_subplots\nimport os\nfrom tqdm import tqdm","metadata":{"execution":{"iopub.status.busy":"2022-04-05T07:34:47.001924Z","iopub.execute_input":"2022-04-05T07:34:47.002195Z","iopub.status.idle":"2022-04-05T07:34:47.008494Z","shell.execute_reply.started":"2022-04-05T07:34:47.002163Z","shell.execute_reply":"2022-04-05T07:34:47.007658Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Defines class for processing train/test images and metadata.\nclass WhaleAndDolphin():\n    def __init__(self, path_to_metadata, path_to_dir_images, \\\n                 path_to_dir_nobg_images, path_to_dir_cropped_images):\n        self._path_to_metadata = path_to_metadata\n        self._path_to_dir_images = path_to_dir_images\n        self._path_to_dir_nobg_images = path_to_dir_nobg_images\n        self._path_to_dir_cropped_images = path_to_dir_cropped_images\n        \n        # Loads metadata to variable \"_metadata_all\"\n        self._metadata_all = pd.read_csv(path_to_metadata)\n        \n        # Adds several colmuns.\n        path_to_images = \\\n            [\"%s/%s\" % (path_to_dir_images, row.image) \\\n             for row in self._metadata_all.itertuples()]\n        self._metadata_all[\"path_to_image\"] = path_to_images\n        \n        path_to_nobg_images = \\\n            [\"%s/%s\" % (path_to_dir_nobg_images, row.image.replace(\".jpg\", \".png\")) \\\n             for row in self._metadata_all.itertuples()]\n        self._metadata_all[\"path_to_nobg_image\"] = path_to_nobg_images\n        \n        path_to_cropped_images = \\\n            [\"%s/%s\" % (path_to_dir_cropped_images, row.image.replace(\".jpg\", \".png\")) \\\n             for row in self._metadata_all.itertuples()]\n        self._metadata_all[\"path_to_cropped_image\"] = path_to_cropped_images\n        \n        annotations_xyxy = \\\n            [[] for row in self._metadata_all.itertuples()]\n        self._metadata_all[\"annotations_xyxy\"] = annotations_xyxy\n        \n        # Copies the metadata for processing it.\n        self._metadata = self._metadata_all.copy()\n        \n        self._all_species = self.getSpecies()\n        self._all_individual_ids = self.getIndividualIDs()\n        \n    def resetMetadata(self, initialize=False):\n        if hasattr(self, \"_metadata_tmp\") and not initialize:\n            self._metadata = self._metadata_tmp.copy()\n        else:\n            self._metadata = self._metadata_all.copy()\n            \n    def saveMetadataTemporary(self):\n        self._metadata_tmp = self._metadata.copy()\n        \n    def filterMetadata(self, query=\"index > -1\"):\n        sliced_metadata = \\\n            self._metadata.query(query).reset_index(drop=True)\n        self._metadata = sliced_metadata.copy()\n        \n    def filterMetadataBackgroundRemovedImageExistence(self):\n        indices = []\n        for row in self._metadata.itertuples():\n            if not os.path.exists(row.path_to_nobg_image):\n                indices.append(row.Index)\n                \n        sliced_metadata = self._metadata.drop(index=indices).reset_index(drop=True)\n        self._metadata = sliced_metadata.copy()\n        \n    def writeMetadata(self, path_to_metadata):\n        self._metadata.to_csv(path_to_metadata, index=False)\n        \n    def getMetadata(self):\n        return self._metadata\n    \n    def getSpecies(self):\n        return self._metadata[\"species\"].unique()\n    \n    def _species2id(self, species):\n        return np.where(self._all_species == species)[0][0]\n    \n    def getIndividualIDs(self):\n        return self._metadata[\"individual_id\"].unique()\n    \n    def showImagesTile(self, num_cols=4, draw_annotations=False):\n        metadata = self._metadata\n        titles = [row.image for row in metadata.itertuples()]\n        path_to_images = [row.path_to_image \\\n                          for row in metadata.itertuples()]\n        images = myutils.getImages(path_to_images)\n        if \"annotations_xyxy\" in metadata.columns and draw_annotations:\n            annotations_xyxy_for_images = [row.annotations_xyxy \\\n                                           for row in metadata.itertuples()]\n            texts_for_images = [[\"\" for _ in \\\n                                 range(len(row.annotations_xyxy))] \\\n                                 for row in metadata.itertuples()]\n            myutils.drawAnnotations( \\\n                images, \\\n                annotations_xyxy_for_images=annotations_xyxy_for_images, \\\n                texts_for_images=texts_for_images, \\\n                line_color=\"green\", line_width=3, text_color=\"green\" \\\n            )\n        myutils.showImagesTile(titles, images, num_cols=num_cols)\n        \n    def showProcessedImagesTile(self, num_cols=3, draw_annotations=False):\n        metadata = self._metadata\n        titles, path_to_images = [], []\n        for row in metadata.itertuples():\n            titles.append(\"%s (Org.)\" % row.image)\n            path_to_images.append(row.path_to_image)\n            \n            titles.append(\"%s (BG. removed)\" % row.image)\n            path_to_images.append(row.path_to_nobg_image)\n            \n            titles.append(\"%s (Cropped)\" % row.image)\n            path_to_images.append(row.path_to_cropped_image)\n        \n        images = myutils.getImages(path_to_images)\n        if \"annotations_xyxy\" in metadata.columns and draw_annotations:\n            annotations_xyxy_for_images = [row.annotations_xyxy \\\n                                           for row in metadata.itertuples()]\n            texts_for_images = [[\"\" for _ in \\\n                                 range(len(row.annotations_xyxy))] \\\n                                 for row in metadata.itertuples()]\n            myutils.drawAnnotations( \\\n                images[::3], \\\n                annotations_xyxy_for_images=annotations_xyxy_for_images, \\\n                texts_for_images=texts_for_images, \\\n                line_color=\"red\", line_width=3, text_color=\"red\" \\\n            ) # For only org. images.\n        myutils.showImagesTile(titles, images, num_cols=num_cols)\n        \n    def showIndividualImagesTile(self, num_cols=4, \\\n                                 max_num_individual_images=4, \\\n                                 max_num_individuals=10, \\\n                                 draw_annotations=False):\n        self.saveMetadataTemporary()\n        \n        individual_ids = self.getIndividualIDs()\n        for individual_id in individual_ids[:max_num_individuals]:\n            print()\n            print(\"Individual ID : %s\" % individual_id)\n            self.filterMetadata(query=\"individual_id == \\\"%s\\\"\" % individual_id)\n            self.filterMetadata(query=\"index < %d\" % max_num_individual_images)\n            self.showImagesTile( \\\n                num_cols=num_cols, \\\n                draw_annotations=draw_annotations\n            )\n            self.resetMetadata()\n            \n    def removeBackground(self):\n        metadata = self._metadata\n        path_to_inputs = [row.path_to_image \\\n                          for row in metadata.itertuples()]\n        path_to_outputs = [row.path_to_nobg_image \\\n                           for row in metadata.itertuples()]\n        \n        for path_to_input, path_to_output in \\\n            zip(path_to_inputs, path_to_outputs):\n            !backgroundremover -i {path_to_input} -o {path_to_output}\n        \n    def calculateAnnotationsXyxy(self):\n        batch_size = 100\n        num_batches = len(self._metadata) // batch_size + 1\n        \n        for i_batch in range(num_batches):\n            i_start = i_batch * batch_size\n            i_end = i_start + batch_size\n            metadata = self._metadata.iloc[i_start:i_end]\n            \n            path_to_nobg_images = [row.path_to_nobg_image \\\n                                   for row in metadata.itertuples()]\n            nobg_images = myutils.getImages(path_to_nobg_images)\n            #fixed_images = [Image.eval(nobg_image, self._removeBugPixel) \\\n            #                for nobg_image in nobg_images]\n            class_ids = [self._species2id(row.species) for row \\\n                         in metadata.itertuples()]\n            \n            annotations_xyxy = []\n            for nobg_image, class_id in zip(nobg_images, class_ids):\n                _, _, _, a = nobg_image.split()\n                x_min, y_min, x_max, y_max = a.getbbox() # Bounding box of non-zero alpha region\n                confidence = 1.0 # Dummy\n                annotation_xyxy = myutils._annotationXyxy(class_id, \\\n                                                          x_min, y_min, \\\n                                                          x_max, y_max, \\\n                                                          confidence)\n                annotations_xyxy.append([annotation_xyxy])\n                \n            self._metadata[\"annotations_xyxy\"].iloc[i_start:i_end] = \\\n                annotations_xyxy\n    \n    #def _removeBugPixel(self, pixel_value):\n    #    if pixel_value == 1:\n    #        return 0\n    #    else:\n    #        return pixel_value\n        \n    def cropObject(self):\n        batch_size = 100\n        num_batches = len(self._metadata) // batch_size + 1\n        \n        for i_batch in range(num_batches):\n            i_start = i_batch * batch_size\n            i_end = i_start + batch_size\n            metadata = self._metadata.iloc[i_start:i_end]\n            \n            path_to_inputs = [row.path_to_image \\\n                              for row in metadata.itertuples()]\n            path_to_outputs = [row.path_to_cropped_image \\\n                               for row in metadata.itertuples()]\n            annotations_xyxy = [row.annotations_xyxy for row \\\n                                in metadata.itertuples()]\n            \n            for path_to_input, path_to_output, annotations_xyxy in \\\n                zip(path_to_inputs, path_to_outputs, annotations_xyxy):\n                \n                image = Image.open(path_to_input)\n                annotation_xyxy = \\\n                    self._maxConfidenceAnnotation(annotations_xyxy)\n                x_min = annotation_xyxy[\"x_min\"]\n                y_min = annotation_xyxy[\"y_min\"]\n                x_max = annotation_xyxy[\"x_max\"]\n                y_max = annotation_xyxy[\"y_max\"]\n                image_cropped = image.crop((x_min, y_min, x_max, y_max))\n                image_cropped.save(path_to_output)\n            \n    def _maxConfidenceAnnotation(self, annotations_xyxy):\n        confidences = np.array([annotation_xyxy[\"confidence\"] \\\n                                for annotation_xyxy in annotations_xyxy])\n        index = np.argmax(confidences)\n        return annotations_xyxy[index]","metadata":{"execution":{"iopub.status.busy":"2022-04-05T07:34:47.010122Z","iopub.execute_input":"2022-04-05T07:34:47.010623Z","iopub.status.idle":"2022-04-05T07:34:47.204659Z","shell.execute_reply.started":"2022-04-05T07:34:47.010580Z","shell.execute_reply":"2022-04-05T07:34:47.203599Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Some of train images are shown as example. Test images can be shown using the same class.","metadata":{}},{"cell_type":"code","source":"# Loads metadata for train images.\nwhale_and_dolphin = WhaleAndDolphin(\n    path_to_metadata=path_to_happywhale_train_metadata,\n    path_to_dir_images=path_to_dir_happywhale_train_images,\n    path_to_dir_nobg_images=path_to_dir_nobg_train_images,\n    path_to_dir_cropped_images=path_to_dir_cropped_train_images\n)","metadata":{"execution":{"iopub.status.busy":"2022-04-05T07:34:47.206551Z","iopub.execute_input":"2022-04-05T07:34:47.207027Z","iopub.status.idle":"2022-04-05T07:34:47.805252Z","shell.execute_reply.started":"2022-04-05T07:34:47.206866Z","shell.execute_reply":"2022-04-05T07:34:47.804321Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Shows train images for the first 3 individuals for each species.\nnum_cols = 4\nmax_num_individual_images = 4\nmax_num_individuals = 3\n\nall_species = whale_and_dolphin.getSpecies()\nfor i, species in enumerate(all_species):\n    whale_and_dolphin.filterMetadata(query=\"species == \\\"%s\\\"\" % species)\n    \n    print()\n    print(\"--------------------------------------------------\")\n    print()\n    print(\"   Images for species No.%02d %s\" % (i, species))\n    print()\n    print(\"--------------------------------------------------\")\n    whale_and_dolphin.showIndividualImagesTile( \\\n        num_cols=num_cols, \\\n        max_num_individual_images=max_num_individual_images, \\\n        max_num_individuals=max_num_individuals \\\n    )\n    print()\n    print()\n    \n    whale_and_dolphin.resetMetadata(initialize=True)","metadata":{"execution":{"iopub.status.busy":"2022-04-05T07:34:47.806708Z","iopub.execute_input":"2022-04-05T07:34:47.807219Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <B>[3] Creation of annotation data</B>","metadata":{}},{"cell_type":"markdown","source":"Annotation data is created with the following steps,\n- Removing background of image,\n- Calculating bounding box,\n- Cropping bounding box,\n- Showing processed images.","metadata":{}},{"cell_type":"markdown","source":"## [3-1] Removing background of image","metadata":{}},{"cell_type":"markdown","source":"Background of image is removed with the command line tool [\"backgroundremover\"](https://github.com/nadermx/backgroundremover) to make easy to calculate bounding box. Sometimes background is not removed properly with the tool, but it works well for almost cases.","metadata":{}},{"cell_type":"markdown","source":"It is difficult to remove background with the tool in a kernel because of RAM utilization limitation of kaggle notebook (~4000 images can be processed in a kernel within the limitation.). So it is done in the series of the following kernels,\n- [\"Whale&Dolphin Background Removed Images (1/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-1-4),\n- [\"Whale&Dolphin Background Removed Images (2/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-2-4),\n- [\"Whale&Dolphin Background Removed Images (3/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-3-4),\n- [\"Whale&Dolphin Background Removed Images (4/4)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-background-removed-images-4-4).","metadata":{}},{"cell_type":"markdown","source":"First, the tool is installed (Background removed images in the other kernels will be used in the kernel, so the code is commented out.).","metadata":{}},{"cell_type":"code","source":"#!pip install backgroundremover","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Then, background is removed (Background removed images in the other kernels will be used in the kernel, so the code is commented out.).","metadata":{}},{"cell_type":"code","source":"# Loads metadata for train images.\nwhale_and_dolphin = WhaleAndDolphin(\n    path_to_metadata=path_to_happywhale_train_metadata,\n    path_to_dir_images=path_to_dir_happywhale_train_images,\n    path_to_dir_nobg_images=path_to_dir_nobg_train_images,\n    path_to_dir_cropped_images=path_to_dir_cropped_train_images\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Removes background. Limits number of processing images because of kaggle notebook limitation. Sets same number for each species as possible.\n#num_images_per_species = int(4000 / len(all_species))\n#i_start = 0\n#i_end = i_start + num_images_per_species\n\n#all_species = whale_and_dolphin.getSpecies()\n#whale_and_dolphin.saveMetadataTemporary()\n\n#for i, species in enumerate(all_species):\n#    whale_and_dolphin.filterMetadata(query=\"species == \\\"%s\\\"\" % species)\n#    whale_and_dolphin.filterMetadata(query=\"%d <= index < %d\" % (i_start, i_end))\n#    print()\n#    print(\"--------------------------------------------------\")\n#    print()\n#    print(\"   Processing images for species No.%02d %s\" % (i, species))\n#    print()\n#    print(\"--------------------------------------------------\")\n#    whale_and_dolphin.removeBackground()\n#    print()\n    \n#    whale_and_dolphin.resetMetadata()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If it succeeds, background removed images are saved in the following directory (Background removed images in the other kernels will be used in the kernel, so the code is commented out.).","metadata":{}},{"cell_type":"code","source":"#!ls {path_to_dir_nobg_train_images} | cat -n | tail -5","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Background removed images in the other kernels can be loaded with the same way as shown in the chapter \"[2-1] Loading dataset\". If it succeeds, the background removed images are loaded to the following path.","metadata":{}},{"cell_type":"code","source":"# Defines path to directories for background removed images.\npath_to_dir_nobg_train_images_part1 = \"/kaggle/input/whale-dolphin-background-removed-images-1-4/nobg/train_images\"\npath_to_dir_nobg_train_images_part2 = \"/kaggle/input/whale-dolphin-background-removed-images-2-4/nobg/train_images\"\npath_to_dir_nobg_train_images_part3 = \"/kaggle/input/whale-dolphin-background-removed-images-3-4/nobg/train_images\"\npath_to_dir_nobg_train_images_part4 = \"/kaggle/input/whale-dolphin-background-removed-images-4-4/nobg/train_images\"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Shows list of background removed images.\n!echo \"{path_to_dir_nobg_train_images_part1}:\"\n!ls {path_to_dir_nobg_train_images_part1} | cat -n | tail -5\n!echo \"\"\n\n!echo \"{path_to_dir_nobg_train_images_part2}:\"\n!ls {path_to_dir_nobg_train_images_part2} | cat -n | tail -5\n!echo \"\"\n\n!echo \"{path_to_dir_nobg_train_images_part3}:\"\n!ls {path_to_dir_nobg_train_images_part3} | cat -n | tail -5\n!echo \"\"\n\n!echo \"{path_to_dir_nobg_train_images_part4}:\"\n!ls {path_to_dir_nobg_train_images_part4} | cat -n | tail -5\n!echo \"\"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"All the images are copied to the working directory.","metadata":{}},{"cell_type":"code","source":"# Copies background removed images into working directories.\n!cp  {path_to_dir_nobg_train_images_part1}/* {path_to_dir_nobg_train_images}\n!cp  {path_to_dir_nobg_train_images_part2}/* {path_to_dir_nobg_train_images}\n!cp  {path_to_dir_nobg_train_images_part3}/* {path_to_dir_nobg_train_images}\n!cp  {path_to_dir_nobg_train_images_part4}/* {path_to_dir_nobg_train_images}","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Shows list of copied background removed images.\n!echo \"{path_to_dir_nobg_train_images}:\"\n!ls {path_to_dir_nobg_train_images} | cat -n | tail -5","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [3-2] Calculating bounding box","metadata":{}},{"cell_type":"markdown","source":"Bounding box is calculated with [\"Image.getbbox()\" of pillow](https://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.Image.getbbox) (\"Image.getbbox()\" is called in the method \"WhaleAndDolphin.calculateAnnotationsXyxy()\".).","metadata":{}},{"cell_type":"code","source":"# Filters metadata, which has background removed image.\nwhale_and_dolphin.filterMetadataBackgroundRemovedImageExistence()\n\nmetadata = whale_and_dolphin.getMetadata()\nmetadata","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Calculates bounding box of object.\nwhale_and_dolphin.calculateAnnotationsXyxy()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If it succeeds, annotation data for showing bounding box is created as follows.","metadata":{}},{"cell_type":"code","source":"# Shows metadata with calculated bounding box.\nmetadata = whale_and_dolphin.getMetadata()\nmetadata","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The metadata with created annotation data is saved as csv file in the following directory.","metadata":{}},{"cell_type":"code","source":"# Saves metadata as csv file.\npath_to_train_metadata = \"/kaggle/working/train_with_annotation.csv\"\nwhale_and_dolphin.writeMetadata(path_to_train_metadata)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Shows created csv file.\n!head {path_to_train_metadata}","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [3-3] Cropping bounding box","metadata":{}},{"cell_type":"markdown","source":"Bounding box is cropped for future work (The model for identifing whale and dolphin may be created using the cropped images.) with [\"Image.crop()\" of pillow](https://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.Image.crop) (\"Image.crop()\" is called in the method \"WhaleAndDolphin.cropObject()\".).","metadata":{}},{"cell_type":"markdown","source":"It is difficult to crop bounding box in a kernel because of disc space utilization limitation of kaggle notebook (~5000 images can be processed in a kernel within the limitation.). So it is done in the series of the following kernels,\n- [\"Whale&Dolphin Cropped Images (1/3)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-cropped-images-1-3) (This kernel),\n- [\"Whale&Dolphin Cropped Images (2/3)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-cropped-images-2-3),\n- [\"Whale&Dolphin Cropped Images (3/3)\"](https://www.kaggle.com/code/acchiko/whale-dolphin-cropped-images-3-3).","metadata":{}},{"cell_type":"code","source":"# Crops object. Limits number of processing images because of disc space utilization limitation.\nnum_images = 4000\ni_start = 0 * num_images\ni_end = i_start + num_images\n\nwhale_and_dolphin.filterMetadata(query=\"%d <= index < %d\" % (i_start, i_end))\nwhale_and_dolphin.cropObject()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If it succeeds, cropped images are saved in the following directory.","metadata":{}},{"cell_type":"code","source":"# Shows list of created cropped images.\n!ls {path_to_dir_cropped_train_images} | cat -n | tail -5","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [3-4] Showing processed images","metadata":{}},{"cell_type":"markdown","source":"Processed images (Original/Background removed/Cropped images) are shown.","metadata":{}},{"cell_type":"code","source":"# Shows first 2000 processed images.\nnum_images_per_batch = 100\nmetadata = whale_and_dolphin.getMetadata()\nnum_batches = int(len(metadata) / num_images_per_batch)\nmax_num_batches = min(num_batches, 20)\nnum_cols = 3\n\nwhale_and_dolphin.saveMetadataTemporary()\nfor i in range(max_num_batches):\n    i_start = i * num_images_per_batch\n    i_end = i_start + num_images_per_batch\n    whale_and_dolphin.filterMetadata(query=\"%d <= index < %d\" % (i_start, i_end))\n    whale_and_dolphin.showProcessedImagesTile( \\\n        num_cols=num_cols, \\\n        draw_annotations=True \\\n    )\n    whale_and_dolphin.resetMetadata()","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}