{"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":91844,"databundleVersionId":11361821,"sourceType":"competition"}],"dockerImageVersionId":30918,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# birdCLEF+ 2025: Species Analysis","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"This year's [birdCLEF+ 2025 competition](https://www.kaggle.com/competitions/birdclef-2025) is about much more than birds. There are 206 classes which besides 70% birds also feature frogs, insects and even jaguars. To build effective models it makes sense to first analyze what kind of animals are featured. We will group species into types. We will also analyze some of the notable sounds which are featured in the dataset.","metadata":{}},{"cell_type":"markdown","source":"## Table of Contents\n\n- [Taxonomy](#Taxonomy)\n- [Types](#Types)\n- [Sounds](#Sounds)\n- [Conclusion](#Conclusion)\n","metadata":{}},{"cell_type":"code","source":"# Render Plotly as png\n!pip install -Uqq kaleido","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-14T16:28:52.292924Z","iopub.execute_input":"2025-03-14T16:28:52.293394Z","iopub.status.idle":"2025-03-14T16:28:56.674676Z","shell.execute_reply.started":"2025-03-14T16:28:52.293349Z","shell.execute_reply":"2025-03-14T16:28:56.673236Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport soundfile as sf\nimport plotly.express as px\nfrom IPython.display import Audio\n\n# Fix Plotly rendering in Jupyter forks.\n# If you are running this notebook locally you can comment this out.\n# This allows you to play with interactive Plotly plots.\nimport plotly.io as pio\npio.renderers.default = 'png'","metadata":{"_kg_hide-input":true,"trusted":true,"execution":{"iopub.status.busy":"2025-03-14T16:28:56.676286Z","iopub.execute_input":"2025-03-14T16:28:56.676754Z","iopub.status.idle":"2025-03-14T16:28:56.682460Z","shell.execute_reply.started":"2025-03-14T16:28:56.676710Z","shell.execute_reply":"2025-03-14T16:28:56.681472Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Taxonomy ","metadata":{}},{"cell_type":"markdown","source":"The `taxonomy.csv` file contains metadata about all species. The columns to focus on are:\n- `primary_label`: Primary id for class also used in the submission file.\n- `scientific_name`: Name as used in the scientific community. Papers and pre-trained models will probably use the scientific names as class names.\n- `common_name`: Straightforward names like \"Mountain Lion\" or \"Blue-headed Parrot\".\n- `class_name`: There are four classes: \n - `Aves` (birds)\n - `Amphibia` (amphibians like frogs)\n - `Insecta` (insects like crickets)\n - `Mammalia` (mammals like monkeys)\n","metadata":{}},{"cell_type":"code","source":"BASE_PATH = \"/kaggle/input/birdclef-2025/\"\nt = pd.read_csv(BASE_PATH + \"taxonomy.csv\")\nt = t.drop(columns=[\"inat_taxon_id\"])\nt.tail(2)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-14T16:28:56.683634Z","iopub.execute_input":"2025-03-14T16:28:56.683963Z","iopub.status.idle":"2025-03-14T16:28:56.717355Z","shell.execute_reply.started":"2025-03-14T16:28:56.683930Z","shell.execute_reply":"2025-03-14T16:28:56.716205Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Classes ","metadata":{}},{"cell_type":"code","source":"print(f\"There are '{len(t)}' different species in the dataset.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-14T16:28:56.722051Z","iopub.execute_input":"2025-03-14T16:28:56.722422Z","iopub.status.idle":"2025-03-14T16:28:56.727998Z","shell.execute_reply.started":"2025-03-14T16:28:56.722395Z","shell.execute_reply":"2025-03-14T16:28:56.726618Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Below you can see that the dataset still consists mainly of birds. 146 species (71%) are birds. However, 60 species (29%) are not birds, which is a substantial amount.","metadata":{}},{"cell_type":"code","source":"def plot_value_counts(value_counts, title, x_label=\"Count\", y_label=\"Category\", top_n=None):\n sorted_counts = value_counts.sort_values()\n sorted_counts = sorted_counts.tail(top_n) if top_n is not None else sorted_counts\n fig = px.bar(\n x=sorted_counts.values,\n y=sorted_counts.index,\n orientation='h',\n labels={'x': x_label, 'y': y_label},\n title=title,\n text=sorted_counts.values\n )\n fig.update_layout(\n font=dict(size=14),\n plot_bgcolor='white',\n hoverlabel=dict(bgcolor=\"white\", font_size=14),\n margin=dict(l=20, r=20, t=40, b=20),\n )\n fig.update_traces(\n texttemplate='%{x}',\n textposition='outside',\n hovertemplate='%{y}
Count: %{x}'\n )\n return fig","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-14T16:28:56.732630Z","iopub.execute_input":"2025-03-14T16:28:56.733047Z","iopub.status.idle":"2025-03-14T16:28:56.748203Z","shell.execute_reply.started":"2025-03-14T16:28:56.733016Z","shell.execute_reply":"2025-03-14T16:28:56.747067Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"fig = plot_value_counts(\n t['class_name'].value_counts(), \n title=\"Distribution of Species by Class\",\n y_label=\"Class Name\"\n)\nfig.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-14T16:28:56.749298Z","iopub.execute_input":"2025-03-14T16:28:56.749595Z","iopub.status.idle":"2025-03-14T16:28:56.892661Z","shell.execute_reply.started":"2025-03-14T16:28:56.749571Z","shell.execute_reply":"2025-03-14T16:28:56.891582Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Types ","metadata":{}},{"cell_type":"markdown","source":"Let's take a closer look at the types of animals. We'll parse phrases like \"frog\" and \"cricket\" from the common names to get a better understanding.","metadata":{}},{"cell_type":"code","source":"def get_types(t):\n phrases = ['frog', 'otter', 'lion', 'fox', 'owl', 'flycatcher', 'hawk', 'parrot', 'toucan', 'raccoon', 'jaguar', \n 'squirrel', 'monkey', 'toad', 'duck', 'kingfisher', 'vulture', 'parakeet', 'dove', 'woodpecker', \n 'blackbird', 'caracara', 'cricket', 'macaw', 'ibis', 'oriol', 'warbler', 'swallow', 'kingbird', \n 'kestrel', 'seedeater', 'spinetail', 'peccary', 'sloth', 'spoonbill', 'hummingbird', 'stork', \n 'cicadas', 'antbird', 'heron', 'sandpiper', 'woodcreeper', 'cuckoo', 'wren', 'falcon', 'tyrant',\n 'katydid', 'anhinga', 'bananaquit', 'donacobius', 'grassquit', 'antshrike', 'jay', 'tityra', \n 'curassow', 'motmot', 'tanager', 'saltator', 'grackle', 'becard', 'aracari', 'chachalaca', \n 'pauraque', 'potoo', 'bobwhite', 'guan', 'oropendola', 'manakin', 'ani', 'egret', 'kiskadee', \n 'tinamou', 'martin', 'tern', 'grebe', 'cormorant', 'screamer', 'piculet', 'hornero', 'pigeon', \n 'puffbird', 'kite', 'gallinule', 'jacamar', 'finch', 'tyrannulet', 'lapwing', 'euphonia', \n 'schiffornis', 'parula', 'jacana', 'trogon', 'elaenia', 'cacique']\n \n def get_names(t, phrase):\n return [i.lower() for i in t['common_name'] if phrase in i.lower()]\n\n result = {p: len(get_names(t, p)) for p in phrases}\n other_names = [name for name in t['common_name'] if not any(p in name.lower() for p in phrases)]\n result['other'] = len(other_names)\n return pd.Series(result)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-14T16:28:56.893662Z","iopub.execute_input":"2025-03-14T16:28:56.893965Z","iopub.status.idle":"2025-03-14T16:28:56.902750Z","shell.execute_reply.started":"2025-03-14T16:28:56.893941Z","shell.execute_reply":"2025-03-14T16:28:56.901331Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"First we'll look at all species grouped using the common name of the species.","metadata":{}},{"cell_type":"code","source":"all_types = get_types(t)\nplot_value_counts(all_types, \"All Species\", top_n=20)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-14T16:28:56.903948Z","iopub.execute_input":"2025-03-14T16:28:56.904280Z","iopub.status.idle":"2025-03-14T16:28:57.069529Z","shell.execute_reply.started":"2025-03-14T16:28:56.904241Z","shell.execute_reply":"2025-03-14T16:28:57.068483Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"We can see that there are quite a few frogs in the dataset. For example, the **Spotted Foam-Nest Frog (label: 126247)**.\n\n","metadata":{}},{"cell_type":"markdown","source":"`other` signifies species where the common name is actually the scientific name. It looks like these are relatively rare species that don't really have a common name. There are 14 of these cases. Most of them are insects, like the **Neoconocephalus brachypterus (label: 1346504)**.\n\n","metadata":{}},{"cell_type":"markdown","source":"If we ignore the `other` category, the 2nd most common species is the **flycatcher**, a type of bird. There are 13 different flycatchers in the dataset. One of them is the **Boat-billed Flycatcher (label: bobfly1)**.\n\n","metadata":{}},{"cell_type":"markdown","source":"Besides the common flycatchers, the bird species also feature **owls**, **woodpeckers**, **parrots** and **hawks**.","metadata":{}},{"cell_type":"code","source":"bird_types = get_types(t[t['class_name'] == 'Aves'])\nplot_value_counts(bird_types, \"Bird Species\", top_n=15)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-14T16:28:57.070542Z","iopub.execute_input":"2025-03-14T16:28:57.070869Z","iopub.status.idle":"2025-03-14T16:28:57.221025Z","shell.execute_reply.started":"2025-03-14T16:28:57.070841Z","shell.execute_reply":"2025-03-14T16:28:57.219845Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"For the non-bird species we have mostly **frogs**, but also mammals which generally make distinct sounds, like **raccoons**, **sloths**, **monkeys**, **squirrels** and **jaguars**. In the data there are also insects that make a high-pitched continuous noise, like **cicadas**.","metadata":{}},{"cell_type":"code","source":"not_bird_types = get_types(t[t['class_name'] != 'Aves'])\nplot_value_counts(not_bird_types, \"Non-Bird Species\", top_n=15)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-14T16:28:57.222108Z","iopub.execute_input":"2025-03-14T16:28:57.222467Z","iopub.status.idle":"2025-03-14T16:28:57.362502Z","shell.execute_reply.started":"2025-03-14T16:28:57.222438Z","shell.execute_reply":"2025-03-14T16:28:57.361435Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Sounds","metadata":{}},{"cell_type":"markdown","source":"Don't be fooled by thinking that groups of species all make similar sounds. For example, in the frog category we have the **Whistling Grass Frog (label: 22973)** which sounds like a bird.\n\n","metadata":{}},{"cell_type":"code","source":"def load_audio(path, sec=None, sample_rate=32_000) -> np.array:\n with sf.SoundFile(path) as f: audio = f.read(int(sec * sample_rate)) if sec else f.read()\n return audio\n\ndef show_audio(path, sec=None):\n return Audio(load_audio(path, sec=sec), rate=32_000)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-14T16:28:57.363529Z","iopub.execute_input":"2025-03-14T16:28:57.363895Z","iopub.status.idle":"2025-03-14T16:28:57.369218Z","shell.execute_reply.started":"2025-03-14T16:28:57.363868Z","shell.execute_reply":"2025-03-14T16:28:57.368072Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"show_audio(BASE_PATH + \"train_audio/22973/XC882793.ogg\", sec=5)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-14T16:28:57.370352Z","iopub.execute_input":"2025-03-14T16:28:57.370774Z","iopub.status.idle":"2025-03-14T16:28:57.413673Z","shell.execute_reply.started":"2025-03-14T16:28:57.370737Z","shell.execute_reply":"2025-03-14T16:28:57.412385Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"There are also animals which make odd sounds, like the **Crab-eating Fox (label: 42087)**. You might be able to create simple heuristics in your inference code to spot these animals.\n\n","metadata":{}},{"cell_type":"markdown","source":"What does the fox say?","metadata":{}},{"cell_type":"code","source":"show_audio(BASE_PATH + \"train_audio/42087/iNat860016.ogg\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-14T16:28:57.414856Z","iopub.execute_input":"2025-03-14T16:28:57.415217Z","iopub.status.idle":"2025-03-14T16:28:57.453447Z","shell.execute_reply.started":"2025-03-14T16:28:57.415187Z","shell.execute_reply":"2025-03-14T16:28:57.452289Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"show_audio(BASE_PATH + \"train_audio/42087/iNat155127.ogg\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-14T16:28:57.454650Z","iopub.execute_input":"2025-03-14T16:28:57.455123Z","iopub.status.idle":"2025-03-14T16:28:57.502304Z","shell.execute_reply.started":"2025-03-14T16:28:57.455079Z","shell.execute_reply":"2025-03-14T16:28:57.501195Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Some insects like the **Typical Cicadas (label: 50186)** make distinct, annoying noises. We should keep this in mind when training models. Can we use heuristics to identify these insects? Will your model generalize well on birds, mammals and insects?\n\n","metadata":{}},{"cell_type":"code","source":"show_audio(BASE_PATH + \"train_audio/50186/CSA35128.ogg\", sec=30)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-14T16:28:57.503682Z","iopub.execute_input":"2025-03-14T16:28:57.504080Z","iopub.status.idle":"2025-03-14T16:28:57.608148Z","shell.execute_reply.started":"2025-03-14T16:28:57.504048Z","shell.execute_reply":"2025-03-14T16:28:57.606997Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Conclusion\n\nWe've seen that even though 70% of the species in the dataset are birds, there are a lot of other types that may require a different approach. We might need to get creative to generalize across all 206 species. Also keep in mind that you can't assume that all animals within the same group sound the same. For example, we have heard that not all frogs sound like frogs.","metadata":{}},{"cell_type":"markdown","source":"That's it! Hope you learned something new and are excited to dive into the competition! \n\nIf you like this Kaggle kernel, consider giving an upvote and leaving a comment. Your feedback is very welcome! I will try to implement your suggestions in this kernel.","metadata":{}}]}