How We Identify Whales Thanks to @tedcheese
Hello everyone!
\nI wanted to start a thread about articles and expand it, and hopefully others will share as well, articles to gain knowledge in the field. I have no knowledge in this area, so I downloaded the articles that seemed relevant after reading their summaries.
\nPapers:
\nIntegral Curvature Representation and Matching Algorithms for Identification of Dolphins and Whales - We address the problem of identifying individual cetaceans from images showing the trailing edge of their fins. Given the trailing edge from an unknown individual, we produce a ranking of known individuals from a database. The nicks and notches along the trailing edge define an individualâs unique signature. We define a representation based on integral curvature that is robust to changes in viewpoint and pose, and captures the pattern of nicks and notches in a local neighborhood at multiple scales. We explore two ranking methods that use this representation. The first uses a dynamic programming time-warping algorithm to align two representations, and interprets the alignment cost as a measure of similarity. This algorithm also exploits learned spatial weights to downweight matches from regions of unstable curvature. The second interprets the representation as a feature descriptor. Feature keypoints are defined at the local extrema of the representation. Descriptors for the set of known individuals are stored in a tree structure, which allows us to perform queries given the descriptors from an unknown trailing edge. We evaluate the top-k accuracy on two real-world datasets to demonstrate the effectiveness of the curvature representation, achieving top-1 accuracy scores of approximately 95% and 80% for bottlenose dolphins and humpback whales, respectively.
Unsupervised Threshold for Automatic Extraction of Dolphin Dorsal Fin Outlines from Digital Photographs in DARWIN (Digital Analysis and Recognition of Whale Images on a Network) - At least two software packages---DARWIN, Eckerd College, and FinScan, Texas A&M---exist to facilitate the identification of cetaceans---whales, dolphins, porpoises---based upon the naturally occurring features along the edges of their dorsal fins. Such identification is useful for biological studies of population, social interaction, migration, etc. The process whereby fin outlines are extracted in current fin-recognition software packages is manually intensive and represents a major user input bottleneck: it is both time consuming and visually fatiguing. This research aims to develop automated methods (employing unsupervised thresholding and morphological processing techniques) to extract cetacean dorsal fin outlines from digital photographs thereby reducing manual user input. Ideally, automatic outline generation will improve the overall user experience and improve the ability of the software to correctly identify cetaceans. Various transformations from color to gray space were examined to determine which produced a grayscale image in which a suitable threshold could be easily identified. To assist with unsupervised thresholding, a new metric was developed to evaluate the jaggedness of figures (\"pixelarity\") in an image after thresholding. The metric indicates how cleanly a threshold segments background and foreground elements and hence provides a good measure of the quality of a given threshold. This research results in successful extractions in roughly 93% of images, and significantly reduces user-input time.
Individual common dolphin identification via metric embedding learning - Photo-identification (photo-id) of dolphin individuals is a commonly used technique in ecological sciences to monitor state and health of individuals, as well as to study the social structure and distribution of a population. Traditional photo-id involves a laborious manual process of matching each dolphin fin photograph captured in the field to a catalogue of known individuals. We examine this problem in the context of open-set recognition and utilise a triplet loss function to learn a compact representation of fin images in a Euclidean embedding, where the Euclidean distance metric represents fin similarity. We show that this compact representation can be successfully learnt from a fairly small (in deep learning context) training set and still generalise well to out-of-sample identities (completely new dolphin individuals), with top-1 and top-5 test set (37 individuals) accuracy of 90.5±2 and 93.6±1 percent. In the presence of 1200 distractors, top-1 accuracy dropped by 12%; however, top-5 accuracy saw only a 2.8% drop.
WHAT IDENTIFIES A WHALE BY ITS FLUKE? ON THE BENEFIT OF INTERPRETABLE MACHINE LEARNING FOR WHALE IDENTIFICATION - Interpretable and explainable machine learning have proven to be promising approaches to verify the quality of a data-driven model in general as well as to obtain more information about the quality of certain observations in practise. In this paper, we use these approaches for an application in the marine sciences to support the monitoring of whales. Whale population monitoring is an important element of whale conservation, where the identification of whales plays an important role in this process, for example to trace the migration of whales over time and space. Classical approaches use photographs and a manual mapping with special focus on the shape of the whale flukes and their unique pigmentation. However, this is not feasible for comprehensive monitoring. Machine learning methods, especially deep neural networks, have shown that they can efficiently solve the automatic observation of a large number of whales. Despite their success for many different tasks such as identification, further potentials such as interpretability and their benefits have not yet been exploited. Our main contribution is an analysis of interpretation tools, especially occlusion sensitivity maps, and the question of how the gained insights can help a whale researcher. For our analysis, we use images of humpback whale flukes provided by the Kaggle Challenge âHumpback Whale Identificationâ. By means of spectral cluster analysis of heatmaps, which indicate which parts of the image are important for a decision, we can show that the they can be grouped in a meaningful way. Moreover, it appears that characteristics automatically determined by a neural network correspond to those that are considered important by a whale expert.
Humpback Whale Identification Challenge: An Overview of the Top Solutions
NDD20: A large-scale few-shot dolphin dataset for coarse and fine-grained\ncategorisation - We introduce the Northumberland Dolphin Dataset 2020 (NDD20), a challenging image dataset annotated for both coarse and fine-grained instance segmentation and categorisation. This dataset, the first release of the NDD, was created in response to the rapid expansion of computer vision into conservation research and the production of field-deployable systems suited to extreme environmental conditions -- an area with few open source datasets. NDD20 contains a large collection of above and below water images of two different dolphin species for traditional coarse and fine-grained segmentation. All data contained in NDD20 was obtained via manual collection in the North Sea around the Northumberland coastline, UK. We present experimentation using standard deep learning network architecture trained using NDD20 and report baselines results.
The Northumberland Dolphin Dataset: A Multimedia Individual Cetacean Dataset for Fine-Grained Categorisation - Methods for cetacean research include photo-identification (photo-id) and passive acoustic monitoring (PAM) which generate thousands of images per expedition that are currently hand categorised by researchers into the individual dolphins sighted. With the vast amount of data obtained it is crucially important to develop a system that is able to categorise this quickly. The Northumberland Dolphin Dataset (NDD) is an on-going novel dataset project made up of above and below water images of, and spectrograms of whistles from, white-beaked dolphins. These are produced by photo-id and PAM data collection methods applied off the coast of Northumberland, UK. This dataset will aid in building cetacean identification models, reducing the number of human-hours required to categorise images. Example use cases and areas identified for speed up are examined.
Combined Color Semantics and Deep Learning for the Automatic Detection of Dolphin Dorsal Fins - Photo-identification is a widely used non-invasive technique in biological studies for understanding if a specimen has been seen multiple times only relying on specific unique visual characteristics. This information is essential to infer knowledge about the spatial distribution, site fidelity, abundance or habitat use of a species. Today there is a large demand for algorithms that can help domain experts in the analysis of large image datasets. For this reason, it is straightforward that the problem of identify and crop the relevant portion of an image is not negligible in any photo-identification pipeline. This paper approaches the problem of automatically cropping cetaceans images with a hybrid technique based on domain analysis and deep learning. Domain knowledge is applied for proposing relevant regions with the aim of highlighting the dorsal fins, then a binary classification of fin vs. no-fin is performed by a convolutional neural network. Results obtained on real images demonstrate the feasibility of the proposed approach in the automated process of large datasets of Rissoâs dolphins photos, enabling its use on more complex large scale studies. Moreover, the results of this study suggest to extend this methodology to biological investigations of different species.
FINâPRINT a fullyâautomated multiâstage deepâlearningâbased framework for the individual recognition of killer whales - Biometric identification techniques such as photoâidentification require an array of unique natural markings to identify individuals. From 1975 to present, Biggâs killer whales have been photoâidentified along the west coast of North America, resulting in one of the largest and longestârunning cetacean photoâidentification datasets. However, data maintenance and analysis are extremely time and resource consuming. This study transfers the procedure of killer whale image identification into a fully automated, multiâstage, deep learning framework, entitled FINâPRINT. It is composed of multiple sequentially ordered subâcomponents. FINâPRINT is trained and evaluated on a dataset collected over an 8âyear period (2011â2018) in the coastal waters off western North America, including 121,000 humanâannotated identification images of Biggâs killer whales. At first, object detection is performed to identify unique killer whale markings, resulting in 94.4% recall, 94.1% precision, and 93.4% meanâaverageâprecision (mAP). Second, all previously identified natural killer whale markings are extracted. The third step introduces a data enhancement mechanism by filtering between valid and invalid markings from previous processing levels, achieving 92.8% recall, 97.5%, precision, and 95.2% accuracy. The fourth and final step involves multiâclass individual recognition. When evaluated on the network test set, it achieved an accuracy of 92.5% with 97.2% topâ3 unweighted accuracy (TUA) for the 100 most commonly photoâidentified killer whales. Additionally, the method achieved an accuracy of 84.5% and a TUA of 92.9% when applied to the entire 2018 image collection of the 100 most common killer whales. The source code of FINâPRINT can be adapted to other species and will be publicly available. Thanks to @bsridatta
Pages:
\nHow We Identify Whales Thanks to @tedcheese
Guide:
\n\nHave a good competition and don't hesitate to comment!
\n![\"Whale\"](\"https://i.postimg.cc/bJ61MLSR/adoptawhale-fluke.jpg\")
Hello everyone!
\nI wanted to start a thread about articles and expand it, and hopefully others will share as well, articles to gain knowledge in the field. I have no knowledge in this area, so I downloaded the articles that seemed relevant after reading their summaries.
\nPapers:
\nIntegral Curvature Representation and Matching Algorithms for Identification of Dolphins and Whales - We address the problem of identifying individual cetaceans from images showing the trailing edge of their fins. Given the trailing edge from an unknown individual, we produce a ranking of known individuals from a database. The nicks and notches along the trailing edge define an individualâs unique signature. We define a representation based on integral curvature that is robust to changes in viewpoint and pose, and captures the pattern of nicks and notches in a local neighborhood at multiple scales. We explore two ranking methods that use this representation. The first uses a dynamic programming time-warping algorithm to align two representations, and interprets the alignment cost as a measure of similarity. This algorithm also exploits learned spatial weights to downweight matches from regions of unstable curvature. The second interprets the representation as a feature descriptor. Feature keypoints are defined at the local extrema of the representation. Descriptors for the set of known individuals are stored in a tree structure, which allows us to perform queries given the descriptors from an unknown trailing edge. We evaluate the top-k accuracy on two real-world datasets to demonstrate the effectiveness of the curvature representation, achieving top-1 accuracy scores of approximately 95% and 80% for bottlenose dolphins and humpback whales, respectively.
Unsupervised Threshold for Automatic Extraction of Dolphin Dorsal Fin Outlines from Digital Photographs in DARWIN (Digital Analysis and Recognition of Whale Images on a Network) - At least two software packages---DARWIN, Eckerd College, and FinScan, Texas A&M---exist to facilitate the identification of cetaceans---whales, dolphins, porpoises---based upon the naturally occurring features along the edges of their dorsal fins. Such identification is useful for biological studies of population, social interaction, migration, etc. The process whereby fin outlines are extracted in current fin-recognition software packages is manually intensive and represents a major user input bottleneck: it is both time consuming and visually fatiguing. This research aims to develop automated methods (employing unsupervised thresholding and morphological processing techniques) to extract cetacean dorsal fin outlines from digital photographs thereby reducing manual user input. Ideally, automatic outline generation will improve the overall user experience and improve the ability of the software to correctly identify cetaceans. Various transformations from color to gray space were examined to determine which produced a grayscale image in which a suitable threshold could be easily identified. To assist with unsupervised thresholding, a new metric was developed to evaluate the jaggedness of figures (\"pixelarity\") in an image after thresholding. The metric indicates how cleanly a threshold segments background and foreground elements and hence provides a good measure of the quality of a given threshold. This research results in successful extractions in roughly 93% of images, and significantly reduces user-input time.
Individual common dolphin identification via metric embedding learning - Photo-identification (photo-id) of dolphin individuals is a commonly used technique in ecological sciences to monitor state and health of individuals, as well as to study the social structure and distribution of a population. Traditional photo-id involves a laborious manual process of matching each dolphin fin photograph captured in the field to a catalogue of known individuals. We examine this problem in the context of open-set recognition and utilise a triplet loss function to learn a compact representation of fin images in a Euclidean embedding, where the Euclidean distance metric represents fin similarity. We show that this compact representation can be successfully learnt from a fairly small (in deep learning context) training set and still generalise well to out-of-sample identities (completely new dolphin individuals), with top-1 and top-5 test set (37 individuals) accuracy of 90.5±2 and 93.6±1 percent. In the presence of 1200 distractors, top-1 accuracy dropped by 12%; however, top-5 accuracy saw only a 2.8% drop.
WHAT IDENTIFIES A WHALE BY ITS FLUKE? ON THE BENEFIT OF INTERPRETABLE MACHINE LEARNING FOR WHALE IDENTIFICATION - Interpretable and explainable machine learning have proven to be promising approaches to verify the quality of a data-driven model in general as well as to obtain more information about the quality of certain observations in practise. In this paper, we use these approaches for an application in the marine sciences to support the monitoring of whales. Whale population monitoring is an important element of whale conservation, where the identification of whales plays an important role in this process, for example to trace the migration of whales over time and space. Classical approaches use photographs and a manual mapping with special focus on the shape of the whale flukes and their unique pigmentation. However, this is not feasible for comprehensive monitoring. Machine learning methods, especially deep neural networks, have shown that they can efficiently solve the automatic observation of a large number of whales. Despite their success for many different tasks such as identification, further potentials such as interpretability and their benefits have not yet been exploited. Our main contribution is an analysis of interpretation tools, especially occlusion sensitivity maps, and the question of how the gained insights can help a whale researcher. For our analysis, we use images of humpback whale flukes provided by the Kaggle Challenge âHumpback Whale Identificationâ. By means of spectral cluster analysis of heatmaps, which indicate which parts of the image are important for a decision, we can show that the they can be grouped in a meaningful way. Moreover, it appears that characteristics automatically determined by a neural network correspond to those that are considered important by a whale expert.
Humpback Whale Identification Challenge: An Overview of the Top Solutions
NDD20: A large-scale few-shot dolphin dataset for coarse and fine-grained\ncategorisation - We introduce the Northumberland Dolphin Dataset 2020 (NDD20), a challenging image dataset annotated for both coarse and fine-grained instance segmentation and categorisation. This dataset, the first release of the NDD, was created in response to the rapid expansion of computer vision into conservation research and the production of field-deployable systems suited to extreme environmental conditions -- an area with few open source datasets. NDD20 contains a large collection of above and below water images of two different dolphin species for traditional coarse and fine-grained segmentation. All data contained in NDD20 was obtained via manual collection in the North Sea around the Northumberland coastline, UK. We present experimentation using standard deep learning network architecture trained using NDD20 and report baselines results.
The Northumberland Dolphin Dataset: A Multimedia Individual Cetacean Dataset for Fine-Grained Categorisation - Methods for cetacean research include photo-identification (photo-id) and passive acoustic monitoring (PAM) which generate thousands of images per expedition that are currently hand categorised by researchers into the individual dolphins sighted. With the vast amount of data obtained it is crucially important to develop a system that is able to categorise this quickly. The Northumberland Dolphin Dataset (NDD) is an on-going novel dataset project made up of above and below water images of, and spectrograms of whistles from, white-beaked dolphins. These are produced by photo-id and PAM data collection methods applied off the coast of Northumberland, UK. This dataset will aid in building cetacean identification models, reducing the number of human-hours required to categorise images. Example use cases and areas identified for speed up are examined.
Combined Color Semantics and Deep Learning for the Automatic Detection of Dolphin Dorsal Fins - Photo-identification is a widely used non-invasive technique in biological studies for understanding if a specimen has been seen multiple times only relying on specific unique visual characteristics. This information is essential to infer knowledge about the spatial distribution, site fidelity, abundance or habitat use of a species. Today there is a large demand for algorithms that can help domain experts in the analysis of large image datasets. For this reason, it is straightforward that the problem of identify and crop the relevant portion of an image is not negligible in any photo-identification pipeline. This paper approaches the problem of automatically cropping cetaceans images with a hybrid technique based on domain analysis and deep learning. Domain knowledge is applied for proposing relevant regions with the aim of highlighting the dorsal fins, then a binary classification of fin vs. no-fin is performed by a convolutional neural network. Results obtained on real images demonstrate the feasibility of the proposed approach in the automated process of large datasets of Rissoâs dolphins photos, enabling its use on more complex large scale studies. Moreover, the results of this study suggest to extend this methodology to biological investigations of different species.
FINâPRINT a fullyâautomated multiâstage deepâlearningâbased framework for the individual recognition of killer whales - Biometric identification techniques such as photoâidentification require an array of unique natural markings to identify individuals. From 1975 to present, Biggâs killer whales have been photoâidentified along the west coast of North America, resulting in one of the largest and longestârunning cetacean photoâidentification datasets. However, data maintenance and analysis are extremely time and resource consuming. This study transfers the procedure of killer whale image identification into a fully automated, multiâstage, deep learning framework, entitled FINâPRINT. It is composed of multiple sequentially ordered subâcomponents. FINâPRINT is trained and evaluated on a dataset collected over an 8âyear period (2011â2018) in the coastal waters off western North America, including 121,000 humanâannotated identification images of Biggâs killer whales. At first, object detection is performed to identify unique killer whale markings, resulting in 94.4% recall, 94.1% precision, and 93.4% meanâaverageâprecision (mAP). Second, all previously identified natural killer whale markings are extracted. The third step introduces a data enhancement mechanism by filtering between valid and invalid markings from previous processing levels, achieving 92.8% recall, 97.5%, precision, and 95.2% accuracy. The fourth and final step involves multiâclass individual recognition. When evaluated on the network test set, it achieved an accuracy of 92.5% with 97.2% topâ3 unweighted accuracy (TUA) for the 100 most commonly photoâidentified killer whales. Additionally, the method achieved an accuracy of 84.5% and a TUA of 92.9% when applied to the entire 2018 image collection of the 100 most common killer whales. The source code of FINâPRINT can be adapted to other species and will be publicly available. Thanks to @bsridatta
Pages:
\nHow We Identify Whales Thanks to @tedcheese
Guide:
\n\nHave a good competition and don't hesitate to comment!
\n
I think, firstly, we should read about how the professionals indetificate the whales, of course understandable, that their fins are different, but I think there are a lot of hidden characteristics! Thank you for sharing, @datascientistfp!
", "rawMarkdown": "I think, firstly, we should read about how the professionals indetificate the whales, of course understandable, that their fins are different, but I think there are a lot of hidden characteristics! Thank you for sharing, @datascientistfp!", "votes": null }, { "id": "1674507", "postDate": "02/03/2022 14:57:22", "content": "Here is an excellent page about identification of gray whales, from one of our data-contributing collaborators, Dr Leigh Torres of Oregon State University: https://www.individuwhale.com/how-we-identify-whales/
", "rawMarkdown": "Here is an excellent page about identification of gray whales, from one of our data-contributing collaborators, Dr Leigh Torres of Oregon State University: [https://www.individuwhale.com/how-we-identify-whales/](https://www.individuwhale.com/how-we-identify-whales/)", "votes": null }, { "id": "1675938", "postDate": "02/04/2022 14:52:15", "content": "Wow! thanks for sharing
", "rawMarkdown": "Wow! thanks for sharing", "votes": null }, { "id": "1677873", "postDate": "02/06/2022 04:46:46", "content": "Another interesting paper!
\nFINâPRINT a fullyâautomated multiâstage deepâlearningâbased framework for the individual recognition of killer whales
Excellent work. Upvoted!
", "rawMarkdown": "Excellent work. Upvoted!", "votes": null }, { "id": "1680239", "postDate": "02/07/2022 17:11:52", "content": "Great post, will definitly help understanding the topic. Thanks!
", "rawMarkdown": "Great post, will definitly help understanding the topic. Thanks!", "votes": null } ], "comments": [ { "id": 1674108, "author_name": "vad13irt", "author_url": "", "post_date": "02/03/2022 08:34:52", "content": "I think, firstly, we should read about how the professionals indetificate the whales, of course understandable, that their fins are different, but I think there are a lot of hidden characteristics! Thank you for sharing, @datascientistfp!
", "votes": null, "replies": [] }, { "id": 1674507, "author_name": "tedcheese", "author_url": "", "post_date": "02/03/2022 14:57:22", "content": "Here is an excellent page about identification of gray whales, from one of our data-contributing collaborators, Dr Leigh Torres of Oregon State University: https://www.individuwhale.com/how-we-identify-whales/
", "votes": null, "replies": [] }, { "id": 1675938, "author_name": "pauljef", "author_url": "", "post_date": "02/04/2022 14:52:15", "content": "Wow! thanks for sharing
", "votes": null, "replies": [] }, { "id": 1677873, "author_name": "bsridatta", "author_url": "", "post_date": "02/06/2022 04:46:46", "content": "Another interesting paper!
\nFINâPRINT a fullyâautomated multiâstage deepâlearningâbased framework for the individual recognition of killer whales
Excellent work. Upvoted!
", "votes": null, "replies": [] }, { "id": 1680239, "author_name": "asafkeidar", "author_url": "", "post_date": "02/07/2022 17:11:52", "content": "Great post, will definitly help understanding the topic. Thanks!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1674034": "Hello everyone!\n\nI wanted to start a thread about articles and expand it, and hopefully others will share as well, articles to gain knowledge in the field. I have no knowledge in this area, so I downloaded the articles that seemed relevant after reading their summaries.\n\n**Papers:**\n\n- [Integral Curvature Representation and Matching Algorithms for Identification of Dolphins and Whales](https://arxiv.org/abs/1708.07785) - We address the problem of identifying individual cetaceans from images showing the trailing edge of their fins. Given the trailing edge from an unknown individual, we produce a ranking of known individuals from a database. The nicks and notches along the trailing edge define an individualâs unique signature. We define a representation based on integral curvature that is robust to changes in viewpoint and pose, and captures the pattern of nicks and notches in a local neighborhood at multiple scales. We explore two ranking methods that use this representation. The first uses a dynamic programming time-warping algorithm to align two representations, and interprets the alignment cost as a measure of similarity. This algorithm also exploits learned spatial weights to downweight matches from regions of unstable curvature. The second interprets the representation as a feature descriptor. Feature keypoints are defined at the local extrema of the representation. Descriptors for the set of known individuals are stored in a tree structure, which allows us to perform queries given the descriptors from an unknown trailing edge. We evaluate the top-k accuracy on two real-world datasets to demonstrate the effectiveness of the curvature representation, achieving top-1 accuracy scores of approximately 95% and 80% for bottlenose dolphins and humpback whales, respectively.\n\n- [Unsupervised Threshold for Automatic Extraction of Dolphin Dorsal Fin Outlines from Digital Photographs in DARWIN (Digital Analysis and Recognition of Whale Images on a Network)](https://arxiv.org/abs/1202.4107) - At least two software packages---DARWIN, Eckerd College, and FinScan, Texas A&M---exist to facilitate the identification of cetaceans---whales, dolphins, porpoises---based upon the naturally occurring features along the edges of their dorsal fins. Such identification is useful for biological studies of population, social interaction, migration, etc. The process whereby fin outlines are extracted in current fin-recognition software packages is manually intensive and represents a major user input bottleneck: it is both time consuming and visually fatiguing. This research aims to develop automated methods (employing unsupervised thresholding and morphological processing techniques) to extract cetacean dorsal fin outlines from digital photographs thereby reducing manual user input. Ideally, automatic outline generation will improve the overall user experience and improve the ability of the software to correctly identify cetaceans. Various transformations from color to gray space were examined to determine which produced a grayscale image in which a suitable threshold could be easily identified. To assist with unsupervised thresholding, a new metric was developed to evaluate the jaggedness of figures (\"pixelarity\") in an image after thresholding. The metric indicates how cleanly a threshold segments background and foreground elements and hence provides a good measure of the quality of a given threshold. This research results in successful extractions in roughly 93% of images, and significantly reduces user-input time. \n\n- [Individual common dolphin identification via metric embedding learning](https://arxiv.org/abs/1901.03662) - Photo-identification (photo-id) of dolphin individuals is a commonly used technique in ecological sciences to monitor state and health of individuals, as well as to study the social structure and distribution of a population. Traditional photo-id involves a laborious manual process of matching each dolphin fin photograph captured in the field to a catalogue of known individuals. We examine this problem in the context of open-set recognition and utilise a triplet loss function to learn a compact representation of fin images in a Euclidean embedding, where the Euclidean distance metric represents fin similarity. We show that this compact representation can be successfully learnt from a fairly small (in deep learning context) training set and still generalise well to out-of-sample identities (completely new dolphin individuals), with top-1 and top-5 test set (37 individuals) accuracy of 90.5±2 and 93.6±1 percent. In the presence of 1200 distractors, top-1 accuracy dropped by 12%; however, top-5 accuracy saw only a 2.8% drop.\n\n- [WHAT IDENTIFIES A WHALE BY ITS FLUKE? ON THE BENEFIT OF INTERPRETABLE MACHINE LEARNING FOR WHALE IDENTIFICATION](https://www.researchgate.net/publication/343401524_WHAT_IDENTIFIES_A_WHALE_BY_ITS_FLUKE_ON_THE_BENEFIT_OF_INTERPRETABLE_MACHINE_LEARNING_FOR_WHALE_IDENTIFICATION) - Interpretable and explainable machine learning have proven to be promising approaches to verify the quality of a data-driven model in general as well as to obtain more information about the quality of certain observations in practise. In this paper, we use these approaches for an application in the marine sciences to support the monitoring of whales. Whale population monitoring is an important element of whale conservation, where the identification of whales plays an important role in this process, for example to trace the migration of whales over time and space. Classical approaches use photographs and a manual mapping with special focus on the shape of the whale flukes and their unique pigmentation. However, this is not feasible for comprehensive monitoring. Machine learning methods, especially deep neural networks, have shown that they can efficiently solve the automatic observation of a large number of whales. Despite their success for many different tasks such as identification, further potentials such as interpretability and their benefits have not yet been exploited. Our main contribution is an analysis of interpretation tools, especially occlusion sensitivity maps, and the question of how the gained insights can help a whale researcher. For our analysis, we use images of humpback whale flukes provided by the Kaggle Challenge âHumpback Whale Identificationâ. By means of spectral cluster analysis of heatmaps, which indicate which parts of the image are important for a decision, we can show that the they can be grouped in a meaningful way. Moreover, it appears that characteristics automatically determined by a neural network correspond to those that are considered important by a whale expert.\n\n- [Humpback Whale Identification Challenge: An Overview of the Top Solutions](https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwiYqIeq8eX1AhUG2BoKHTsOCe8QFnoECCYQAQ&url=https%3A%2F%2Fwww.ic.unicamp.br%2F~meidanis%2FPUB%2FIC%2F2019-Simoes%2FHWIC.pdf&usg=AOvVaw19zMF2U7kYgqulauK4Z8i0)\n\n- [NDD20: A large-scale few-shot dolphin dataset for coarse and fine-grained\ncategorisation](https://arxiv.org/abs/2005.13359) - We introduce the Northumberland Dolphin Dataset 2020 (NDD20), a challenging image dataset annotated for both coarse and fine-grained instance segmentation and categorisation. This dataset, the first release of the NDD, was created in response to the rapid expansion of computer vision into conservation research and the production of field-deployable systems suited to extreme environmental conditions -- an area with few open source datasets. NDD20 contains a large collection of above and below water images of two different dolphin species for traditional coarse and fine-grained segmentation. All data contained in NDD20 was obtained via manual collection in the North Sea around the Northumberland coastline, UK. We present experimentation using standard deep learning network architecture trained using NDD20 and report baselines results. \n\n- [The Northumberland Dolphin Dataset: A Multimedia Individual Cetacean Dataset for Fine-Grained Categorisation](https://arxiv.org/abs/1908.02669) - Methods for cetacean research include photo-identification (photo-id) and passive acoustic monitoring (PAM) which generate thousands of images per expedition that are currently hand categorised by researchers into the individual dolphins sighted. With the vast amount of data obtained it is crucially important to develop a system that is able to categorise this quickly. The Northumberland Dolphin Dataset (NDD) is an on-going novel dataset project made up of above and below water images of, and spectrograms of whistles from, white-beaked dolphins. These are produced by photo-id and PAM data collection methods applied off the coast of Northumberland, UK. This dataset will aid in building cetacean identification models, reducing the number of human-hours required to categorise images. Example use cases and areas identified for speed up are examined. \n\n- [Combined Color Semantics and Deep Learning for the Automatic Detection of Dolphin Dorsal Fins](https://www.researchgate.net/publication/341210741_Combined_Color_Semantics_and_Deep_Learning_for_the_Automatic_Detection_of_Dolphin_Dorsal_Fins) - Photo-identification is a widely used non-invasive technique in biological studies for understanding if a specimen has been seen multiple times only relying on specific unique visual characteristics. This information is essential to infer knowledge about the spatial distribution, site fidelity, abundance or habitat use of a species. Today there is a large demand for algorithms that can help domain experts in the analysis of large image datasets. For this reason, it is straightforward that the problem of identify and crop the relevant portion of an image is not negligible in any photo-identification pipeline. This paper approaches the problem of automatically cropping cetaceans images with a hybrid technique based on domain analysis and deep learning. Domain knowledge is applied for proposing relevant regions with the aim of highlighting the dorsal fins, then a binary classification of fin vs. no-fin is performed by a convolutional neural network. Results obtained on real images demonstrate the feasibility of the proposed approach in the automated process of large datasets of Rissoâs dolphins photos, enabling its use on more complex large scale studies. Moreover, the results of this study suggest to extend this methodology to biological investigations of different species.\n\n- [FINâPRINT a fullyâautomated multiâstage deepâlearningâbased framework for the individual recognition of killer whales](https://www.nature.com/articles/s41598-021-02506-6.pdf) - Biometric identification techniques such as photoâidentification require an array of unique natural markings to identify individuals. From 1975 to present, Biggâs killer whales have been photoâidentified along the west coast of North America, resulting in one of the largest and longestârunning cetacean photoâidentification datasets. However, data maintenance and analysis are extremely time and resource consuming. This study transfers the procedure of killer whale image identification into a fully automated, multiâstage, deep learning framework, entitled FINâPRINT. It is composed of multiple sequentially ordered subâcomponents. FINâPRINT is trained and evaluated on a dataset collected over an 8âyear period (2011â2018) in the coastal waters off western North America, including 121,000 humanâannotated identification images of Biggâs killer whales. At first, object detection is performed to identify unique killer whale markings, resulting in 94.4% recall, 94.1% precision, and 93.4% meanâaverageâprecision (mAP). Second, all previously identified natural killer whale markings are extracted. The third step introduces a data enhancement mechanism by filtering between valid and invalid markings from previous processing levels, achieving 92.8% recall, 97.5%, precision, and 95.2% accuracy. The fourth and final step involves multiâclass individual recognition. When evaluated on the network test set, it achieved an accuracy of 92.5% with 97.2% topâ3 unweighted accuracy (TUA) for the 100 most commonly photoâidentified killer whales. Additionally, the method achieved an accuracy of 84.5% and a TUA of 92.9% when applied to the entire 2018 image collection of the 100 most common killer whales. The source code of FINâPRINT can be adapted to other species and will be publicly available. **Thanks to @bsridatta**\n\n**Pages:**\n- [How We Identify Whales](https://www.individuwhale.com/how-we-identify-whales/) **Thanks to @tedcheese**\n\n- [Identifying whales at sea](https://www.awe.gov.au/environment/marine/marine-species/cetaceans/whale-watching/identification)\n\n\n**Guide:**\n- [Whale, dolphin & porpoise identification guide](https://wildwhales.org/wp-content/uploads/2018/01/BCCSN_IDGuide.pdf)\n\n**Have a good competition and don't hesitate to comment!**\n", "1674108": "I think, firstly, we should read about how the professionals indetificate the whales, of course understandable, that their fins are different, but I think there are a lot of hidden characteristics! Thank you for sharing, @datascientistfp!", "1674507": "Here is an excellent page about identification of gray whales, from one of our data-contributing collaborators, Dr Leigh Torres of Oregon State University: [https://www.individuwhale.com/how-we-identify-whales/](https://www.individuwhale.com/how-we-identify-whales/)", "1675938": "Wow! thanks for sharing", "1677873": "Another interesting paper!\n[FINâPRINT a fullyâautomated multiâstage deepâlearningâbased framework for the individual recognition of killer whales](https://www.nature.com/articles/s41598-021-02506-6.pdf)", "1678920": "Excellent work. Upvoted!", "1680239": "Great post, will definitly help understanding the topic. Thanks!" }, "source": "meta" }![\"Identification\"](\"https://i.postimg.cc/WpFvzgxy/image014.jpg\")