{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"!pip install chart_studio","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-03-20T03:08:40.387571Z","iopub.execute_input":"2023-03-20T03:08:40.387987Z","iopub.status.idle":"2023-03-20T03:08:47.287741Z","shell.execute_reply.started":"2023-03-20T03:08:40.387951Z","shell.execute_reply":"2023-03-20T03:08:47.286501Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport librosa\nimport librosa.display\n%matplotlib inline\n\n# Preprocessing\nimport sklearn\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import LabelEncoder, StandardScaler\n\nimport datetime as dt\nfrom datetime import datetime   \n\n# Visualisation libraries\nimport matplotlib.pyplot as plt\nimport plotly.graph_objs as go\nimport chart_studio.plotly as py\nimport plotly.figure_factory as ff\nfrom plotly.offline import iplot\nimport cufflinks\ncufflinks.go_offline()\ncufflinks.set_config_file(world_readable=True, theme='pearl')\n\n# Settings for pretty nice plots\nplt.style.use('fivethirtyeight')\nplt.show()\n\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\nimport warnings\nwarnings.filterwarnings('ignore')","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-03-20T03:08:47.293544Z","iopub.execute_input":"2023-03-20T03:08:47.293936Z","iopub.status.idle":"2023-03-20T03:08:53.501832Z","shell.execute_reply.started":"2023-03-20T03:08:47.293901Z","shell.execute_reply":"2023-03-20T03:08:53.500758Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Improving the accuracy of soundscape analyses.\n\n## Objective\n\nIt is often easier to hear birds than see them. With proper sound detection and classification, researchers could automatically intuit factors about an area’s quality of life based on a changing bird population.The objective of the competition as stated on the competitions page is to identify a wide variety of bird vocalizations in soundscape recordings. However, the recordings are complex and may contain anthropogenic sounds (e.g., airplane overflights) or other bird and non-bird (e.g., chipmunk) calls in the background, with a particular labeled bird species in the foreground.\n\n## Understanding the Evaluation Metric\n\nThe metric in this competition is the row-wise micro averaged F1 score.The F1 score or F measure, is a measure of a test’s accuracy.\n\nThe F score is defined as the weighted harmonic mean of the test’s precision and recall.\n![](https://imgur.com/nC4QwrO.png)\n\n\n## Data\nFollowing files have been provided to the participants:\n\n* `train_audio` : Trainign data consisting of short recordings\n* `train_csv` : metadata for training data\n*  `test_audio` : The hidden test set audio consists of approximately 150 recordings in mp3 format, each                       roughly 10 minutes long. \n* `test_audio.csv` : metadata for test set.It is important to note that only the first three rows are available for download; the full test.csv is in the hidden test set.\n\n## Exploring the Training metadata\n\nTo begin with let's explore the training metadata file to gather some information","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv('../input/birdsong-recognition/train.csv',)\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:53.503264Z","iopub.execute_input":"2023-03-20T03:08:53.503572Z","iopub.status.idle":"2023-03-20T03:08:53.852332Z","shell.execute_reply.started":"2023-03-20T03:08:53.503539Z","shell.execute_reply":"2023-03-20T03:08:53.851141Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.info()","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:53.854140Z","iopub.execute_input":"2023-03-20T03:08:53.854488Z","iopub.status.idle":"2023-03-20T03:08:53.906313Z","shell.execute_reply.started":"2023-03-20T03:08:53.854456Z","shell.execute_reply":"2023-03-20T03:08:53.905230Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The file contains a lot of columns but we shall focus on some of the ones which are directly related to our problem","metadata":{}},{"cell_type":"markdown","source":"## e-bird code\n\na code for the bird species. You can review detailed information about the bird codes by appending the code to https://ebird.org/species/, such as https://ebird.org/species/amecro for the American Crow.","metadata":{}},{"cell_type":"code","source":"len(train['ebird_code'].value_counts())","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:53.910920Z","iopub.execute_input":"2023-03-20T03:08:53.911459Z","iopub.status.idle":"2023-03-20T03:08:53.922890Z","shell.execute_reply.started":"2023-03-20T03:08:53.911407Z","shell.execute_reply":"2023-03-20T03:08:53.921781Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x = train['ebird_code'].value_counts().index.to_list()\ne_code_path = 'https://ebird.org/species/skylar'\nspecies = [e_code_path+p for p in x]\n","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:37:37.312053Z","iopub.execute_input":"2023-03-20T03:37:37.312473Z","iopub.status.idle":"2023-03-20T03:37:37.322383Z","shell.execute_reply.started":"2023-03-20T03:37:37.312438Z","shell.execute_reply":"2023-03-20T03:37:37.321294Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's check out a few","metadata":{}},{"cell_type":"code","source":"from IPython.display import IFrame\nIFrame(species[0], width=800, height=450)\n","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:37:51.078396Z","iopub.execute_input":"2023-03-20T03:37:51.078803Z","iopub.status.idle":"2023-03-20T03:37:51.086095Z","shell.execute_reply.started":"2023-03-20T03:37:51.078767Z","shell.execute_reply":"2023-03-20T03:37:51.084699Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"IFrame(species[100], width=800, height=450)\n","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:53.956427Z","iopub.execute_input":"2023-03-20T03:08:53.957235Z","iopub.status.idle":"2023-03-20T03:08:53.972727Z","shell.execute_reply.started":"2023-03-20T03:08:53.957184Z","shell.execute_reply":"2023-03-20T03:08:53.971149Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"IFrame(species[200], width=800, height=450)","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:53.974474Z","iopub.execute_input":"2023-03-20T03:08:53.975188Z","iopub.status.idle":"2023-03-20T03:08:53.987737Z","shell.execute_reply.started":"2023-03-20T03:08:53.975140Z","shell.execute_reply":"2023-03-20T03:08:53.986696Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Recordist\n\nRecordist is the user who provided the recordings.","metadata":{}},{"cell_type":"code","source":"# Total number of people who provided the recordings\ntrain['recordist'].nunique()","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:53.989462Z","iopub.execute_input":"2023-03-20T03:08:53.989867Z","iopub.status.idle":"2023-03-20T03:08:54.001968Z","shell.execute_reply.started":"2023-03-20T03:08:53.989817Z","shell.execute_reply":"2023-03-20T03:08:54.001090Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Top 10 recordists in terms of the number of recordings done\ntrain['recordist'].value_counts()[:10].sort_values().iplot(kind='barh',color='#3780BF')","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:54.003159Z","iopub.execute_input":"2023-03-20T03:08:54.003653Z","iopub.status.idle":"2023-03-20T03:08:54.071623Z","shell.execute_reply.started":"2023-03-20T03:08:54.003615Z","shell.execute_reply":"2023-03-20T03:08:54.069908Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Playback_used\n\nWhether playback has been used or not.","metadata":{}},{"cell_type":"code","source":"train['playback_used'].fillna('Not Defined',inplace=True);\ntrain['playback_used'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:54.073307Z","iopub.execute_input":"2023-03-20T03:08:54.073716Z","iopub.status.idle":"2023-03-20T03:08:54.087190Z","shell.execute_reply.started":"2023-03-20T03:08:54.073654Z","shell.execute_reply":"2023-03-20T03:08:54.085921Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train['playback_used'].value_counts()\n\nlabels = train['playback_used'].value_counts().index\nvalues = train['playback_used'].value_counts().values\ncolors=['#3795bf','#bfbfbf']\n\nfig = go.Figure(data=[go.Pie(labels=labels, values=values, textinfo='label+percent',\n                             insidetextorientation='radial',marker=dict(colors=colors))])\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-20T03:08:54.088508Z","iopub.execute_input":"2023-03-20T03:08:54.088807Z","iopub.status.idle":"2023-03-20T03:08:54.117834Z","shell.execute_reply.started":"2023-03-20T03:08:54.088780Z","shell.execute_reply":"2023-03-20T03:08:54.116794Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Ratings","metadata":{}},{"cell_type":"code","source":"train['rating'].value_counts().iplot(kind='bar',color='#3780BF')","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:54.119219Z","iopub.execute_input":"2023-03-20T03:08:54.119599Z","iopub.status.idle":"2023-03-20T03:08:54.180713Z","shell.execute_reply.started":"2023-03-20T03:08:54.119565Z","shell.execute_reply":"2023-03-20T03:08:54.179420Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Date of recordings","metadata":{}},{"cell_type":"code","source":"# Convert string to datetime64\ntrain['date'] = train['date'].apply(pd.to_datetime,format='%Y-%m-%d', errors='coerce')\n#train.set_index('date',inplace=True)\ntrain['date'].value_counts().plot(figsize=(12,8))","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:54.182200Z","iopub.execute_input":"2023-03-20T03:08:54.182527Z","iopub.status.idle":"2023-03-20T03:08:57.143670Z","shell.execute_reply.started":"2023-03-20T03:08:54.182495Z","shell.execute_reply":"2023-03-20T03:08:57.142765Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Countries where recordings have been made","metadata":{}},{"cell_type":"code","source":"%%HTML\n<div class='tableauPlaceholder' id='viz1592397692077' style='position: relative'><noscript><a href='#'><img alt=' ' src='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;Bi&#47;Birds_15923974075490&#47;Dashboard1&#47;1_rss.png' style='border: none' /></a></noscript><object class='tableauViz'  style='display:none;'><param name='host_url' value='https%3A%2F%2Fpublic.tableau.com%2F' /> <param name='embed_code_version' value='3' /> <param name='site_root' value='' /><param name='name' value='Birds_15923974075490&#47;Dashboard1' /><param name='tabs' value='no' /><param name='toolbar' value='yes' /><param name='static_image' value='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;Bi&#47;Birds_15923974075490&#47;Dashboard1&#47;1.png' /> <param name='animate_transition' value='yes' /><param name='display_static_image' value='yes' /><param name='display_spinner' value='yes' /><param name='display_overlay' value='yes' /><param name='display_count' value='yes' /><param name='language' value='en' /><param name='filter' value='publish=yes' /></object></div>                <script type='text/javascript'>                    var divElement = document.getElementById('viz1592397692077');                    var vizElement = divElement.getElementsByTagName('object')[0];                    if ( divElement.offsetWidth > 800 ) { vizElement.style.minWidth='420px';vizElement.style.maxWidth='650px';vizElement.style.width='100%';vizElement.style.minHeight='587px';vizElement.style.maxHeight='887px';vizElement.style.height=(divElement.offsetWidth*0.75)+'px';} else if ( divElement.offsetWidth > 500 ) { vizElement.style.minWidth='420px';vizElement.style.maxWidth='650px';vizElement.style.width='100%';vizElement.style.minHeight='587px';vizElement.style.maxHeight='887px';vizElement.style.height=(divElement.offsetWidth*0.75)+'px';} else { vizElement.style.width='100%';vizElement.style.height='727px';}                     var scriptElement = document.createElement('script');                    scriptElement.src = 'https://public.tableau.com/javascripts/api/viz_v1.js';                    vizElement.parentNode.insertBefore(scriptElement, vizElement);                </script>","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-20T03:08:57.145224Z","iopub.execute_input":"2023-03-20T03:08:57.145521Z","iopub.status.idle":"2023-03-20T03:08:57.153326Z","shell.execute_reply.started":"2023-03-20T03:08:57.145492Z","shell.execute_reply":"2023-03-20T03:08:57.152209Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Species","metadata":{}},{"cell_type":"code","source":"# Total no of unique species in the dataset\nprint(len(train['species'].value_counts().index))\n","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:57.155176Z","iopub.execute_input":"2023-03-20T03:08:57.155886Z","iopub.status.idle":"2023-03-20T03:08:57.175733Z","shell.execute_reply.started":"2023-03-20T03:08:57.155834Z","shell.execute_reply":"2023-03-20T03:08:57.174527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train['species'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:57.177276Z","iopub.execute_input":"2023-03-20T03:08:57.177750Z","iopub.status.idle":"2023-03-20T03:08:57.195650Z","shell.execute_reply.started":"2023-03-20T03:08:57.177710Z","shell.execute_reply":"2023-03-20T03:08:57.194206Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train['species'].value_counts().iplot()","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:57.197467Z","iopub.execute_input":"2023-03-20T03:08:57.197844Z","iopub.status.idle":"2023-03-20T03:08:57.257833Z","shell.execute_reply.started":"2023-03-20T03:08:57.197800Z","shell.execute_reply":"2023-03-20T03:08:57.256547Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Distribution of Recorded Species","metadata":{}},{"cell_type":"code","source":"%%HTML\n<div class='tableauPlaceholder' id='viz1592442148007' style='position: relative'><noscript><a href='#'><img alt=' ' src='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;ZN&#47;ZNDRZCHNN&#47;1_rss.png' style='border: none' /></a></noscript><object class='tableauViz'  style='display:none;'><param name='host_url' value='https%3A%2F%2Fpublic.tableau.com%2F' /> <param name='embed_code_version' value='3' /> <param name='path' value='shared&#47;ZNDRZCHNN' /> <param name='toolbar' value='yes' /><param name='static_image' value='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;ZN&#47;ZNDRZCHNN&#47;1.png' /> <param name='animate_transition' value='yes' /><param name='display_static_image' value='yes' /><param name='display_spinner' value='yes' /><param name='display_overlay' value='yes' /><param name='display_count' value='yes' /><param name='language' value='en' /><param name='filter' value='publish=yes' /></object></div>                <script type='text/javascript'>                    var divElement = document.getElementById('viz1592442148007');                    var vizElement = divElement.getElementsByTagName('object')[0];                    if ( divElement.offsetWidth > 800 ) { vizElement.style.minWidth='420px';vizElement.style.maxWidth='650px';vizElement.style.width='100%';vizElement.style.minHeight='587px';vizElement.style.maxHeight='887px';vizElement.style.height=(divElement.offsetWidth*0.75)+'px';} else if ( divElement.offsetWidth > 500 ) { vizElement.style.minWidth='420px';vizElement.style.maxWidth='650px';vizElement.style.width='100%';vizElement.style.minHeight='587px';vizElement.style.maxHeight='887px';vizElement.style.height=(divElement.offsetWidth*0.75)+'px';} else { vizElement.style.width='100%';vizElement.style.height='727px';}                     var scriptElement = document.createElement('script');                    scriptElement.src = 'https://public.tableau.com/javascripts/api/viz_v1.js';                    vizElement.parentNode.insertBefore(scriptElement, vizElement);                </script>","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-20T03:08:57.259079Z","iopub.execute_input":"2023-03-20T03:08:57.259384Z","iopub.status.idle":"2023-03-20T03:08:57.270231Z","shell.execute_reply.started":"2023-03-20T03:08:57.259354Z","shell.execute_reply":"2023-03-20T03:08:57.268654Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Additional Datasets\n\nIt is interesting to note here the amount of training recordings per species has been limited to 100 to keep the dataset at a reasonable. However, external dataset is available and [Rohan Rao](https://www.kaggle.com/rohanrao) has been kind enough to extract them and share them in usable form with the community.\n\nThe remaining recordings (with some exclusions) has been published as datasets (split in two by first alphabet due to Kaggle's 20GB size limitation):\nhttp://www.kaggle.com/rohanrao/xeno-canto-bird-recordings-extended-a-m\nhttp://www.kaggle.com/rohanrao/xeno-canto-bird-recordings-extended-n-z\n\nPlease read [this discussion post](https://www.kaggle.com/c/birdsong-recognition/discussion/159970) for more information.\n\n### Analysing the external datasets","metadata":{}},{"cell_type":"code","source":"TRAIN_EXT_PATH = \"../input/xeno-canto-bird-recordings-extended-a-m/train_extended.csv\"\ntrain_ext = pd.read_csv(TRAIN_EXT_PATH)\ntrain_ext.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:57.271536Z","iopub.execute_input":"2023-03-20T03:08:57.271921Z","iopub.status.idle":"2023-03-20T03:08:57.582059Z","shell.execute_reply.started":"2023-03-20T03:08:57.271886Z","shell.execute_reply":"2023-03-20T03:08:57.580829Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(train_ext['ebird_code'].value_counts())","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:57.583504Z","iopub.execute_input":"2023-03-20T03:08:57.583824Z","iopub.status.idle":"2023-03-20T03:08:57.594197Z","shell.execute_reply.started":"2023-03-20T03:08:57.583794Z","shell.execute_reply":"2023-03-20T03:08:57.593224Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(train_ext)","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:08:57.596053Z","iopub.execute_input":"2023-03-20T03:08:57.596405Z","iopub.status.idle":"2023-03-20T03:08:57.609385Z","shell.execute_reply.started":"2023-03-20T03:08:57.596369Z","shell.execute_reply":"2023-03-20T03:08:57.607577Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The original train data has 21,375 recordings and the extended train data has an additional 22,015 recordings for 253 out of 264 species which more than doubles the training data size.","metadata":{}},{"cell_type":"code","source":"df_original = train.groupby(\"species\")[\"filename\"].count().reset_index().rename(columns = {\"filename\": \"original_recordings\"})\ndf_extended = train_ext.groupby(\"species\")[\"filename\"].count().reset_index().rename(columns = {\"filename\": \"extended_recordings\"})\n\ndf = df_original.merge(df_extended, on = \"species\", how = \"left\").fillna(0)\ndf[\"total_recordings\"] = df.original_recordings + df.extended_recordings\ndf = df.sort_values(\"total_recordings\").reset_index().sort_values('total_recordings',ascending=False)\ndf.head()","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-03-20T03:08:57.611245Z","iopub.execute_input":"2023-03-20T03:08:57.611577Z","iopub.status.idle":"2023-03-20T03:08:57.654094Z","shell.execute_reply.started":"2023-03-20T03:08:57.611547Z","shell.execute_reply":"2023-03-20T03:08:57.653098Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Plot the total recordings\nf, ax = plt.subplots(figsize=(10, 50))\n\nsns.set_color_codes(\"pastel\")\nsns.barplot(x=\"total_recordings\", y=\"species\", data=df,\n            label=\"total_recordings\", color=\"r\")\n\n# Plot the original recordings\nsns.set_color_codes(\"muted\")\nsns.barplot(x=\"original_recordings\", y=\"species\", data=df,\n            label=\"original_recordings\", color=\"g\")\n\n# Add a legend and informative axis label\nax.legend(ncol=2, loc=\"lower right\", frameon=True)\nax.set(xlim=(0, 2000), ylabel=\"\",\n       xlabel=\"Count\")\nsns.despine(left=True, bottom=True)","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2023-03-20T03:08:57.655589Z","iopub.execute_input":"2023-03-20T03:08:57.655962Z","iopub.status.idle":"2023-03-20T03:09:02.948162Z","shell.execute_reply.started":"2023-03-20T03:08:57.655929Z","shell.execute_reply":"2023-03-20T03:09:02.947289Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Clearly, the additional recordings(in red) has far more observations than existing data. Since external data is allowed in the competition, it can prove to be highly beneficial during the training part.","metadata":{}},{"cell_type":"markdown","source":"# Properties of an audio signal\n\n\nAn audio signal is represented in the form of an audio signal having parameters such as frequency, bandwidth, decibel etc. A typical audio signal can be expressed as a function of Amplitude and Time.\n![](https://miro.medium.com/max/1400/1*akRbhl8739UEDuKHkOUR1Q.png)\n\nsource: https://docs.google.com/presentation/d/1zzgNu_HbKL2iPkHS8-qhtDV20QfWt9lC3ZwPVZo8Rw0/pub?start=false&loop=false&delayms=3000&slide=id.g5a7a9806e_0_84\n\n\nThese sounds are available in many formats which makes it possible for the computer to read and analyse them. Some examples are:\n* mp3 format\n* WMA (Windows Media Audio) format\n* wav (Waveform Audio File) format","metadata":{}},{"cell_type":"markdown","source":"## Loading an audio file\n\nI'll be using a Python Library called Librosa for analysing the audio file.","metadata":{}},{"cell_type":"code","source":"\naudio_path = '../input/birdsong-recognition/train_audio/nutwoo/XC462016.mp3'\nx , sr = librosa.load(audio_path)\n","metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","execution":{"iopub.status.busy":"2023-03-20T03:09:02.949414Z","iopub.execute_input":"2023-03-20T03:09:02.949921Z","iopub.status.idle":"2023-03-20T03:09:06.671787Z","shell.execute_reply.started":"2023-03-20T03:09:02.949871Z","shell.execute_reply":"2023-03-20T03:09:06.670165Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(type(x), type(sr))","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:06.673997Z","iopub.execute_input":"2023-03-20T03:09:06.674374Z","iopub.status.idle":"2023-03-20T03:09:06.681096Z","shell.execute_reply.started":"2023-03-20T03:09:06.674337Z","shell.execute_reply":"2023-03-20T03:09:06.679767Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(x.shape, sr)","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:06.683165Z","iopub.execute_input":"2023-03-20T03:09:06.683521Z","iopub.status.idle":"2023-03-20T03:09:06.696196Z","shell.execute_reply.started":"2023-03-20T03:09:06.683488Z","shell.execute_reply":"2023-03-20T03:09:06.694912Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This returns an audio time series as a numpy array with a default sampling rate(sr) of 22KHZ mono. sr stands for **Sample Rate** which is the number of samples of audio carried per second, measured in Hz or kHz.\n\nWe can also resample the audio sample to **44.1KHz** by:","metadata":{}},{"cell_type":"code","source":"librosa.load(audio_path, sr=44100)","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:06.698053Z","iopub.execute_input":"2023-03-20T03:09:06.698630Z","iopub.status.idle":"2023-03-20T03:09:07.105784Z","shell.execute_reply.started":"2023-03-20T03:09:06.698386Z","shell.execute_reply":"2023-03-20T03:09:07.104514Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Or we can simply disable sampling by:","metadata":{}},{"cell_type":"code","source":"librosa.load(audio_path, sr=None)","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:07.107708Z","iopub.execute_input":"2023-03-20T03:09:07.108069Z","iopub.status.idle":"2023-03-20T03:09:07.490161Z","shell.execute_reply.started":"2023-03-20T03:09:07.108034Z","shell.execute_reply":"2023-03-20T03:09:07.488673Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Playing Audio\n\n`IPython.display.Audio` lets you play audio directly in a jupyter notebook.","metadata":{}},{"cell_type":"code","source":"import IPython.display as ipd\nipd.Audio(audio_path)","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:07.492206Z","iopub.execute_input":"2023-03-20T03:09:07.492585Z","iopub.status.idle":"2023-03-20T03:09:07.525636Z","shell.execute_reply.started":"2023-03-20T03:09:07.492546Z","shell.execute_reply":"2023-03-20T03:09:07.524322Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualizing Audio\n\nWe can plot the audio array using librosa.display.waveplot. Let's plot the  amplitude envelope of a waveform.\n","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(14, 5))\nlibrosa.display.waveplot(x, sr=sr)","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:07.532495Z","iopub.execute_input":"2023-03-20T03:09:07.533017Z","iopub.status.idle":"2023-03-20T03:09:07.864616Z","shell.execute_reply.started":"2023-03-20T03:09:07.532971Z","shell.execute_reply":"2023-03-20T03:09:07.863757Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here, we have the plot of the amplitude envelope of a waveform.","metadata":{}},{"cell_type":"markdown","source":"### 1. Spectrogram\nA spectrogram is a visual representation of the spectrum of frequencies of sound or other signals as they vary with time. Spectrograms are sometimes called sonographs, voiceprints, or voicegrams. When the data is represented in a 3D plot, they may be called waterfalls. In 2-dimensional arrays, the first axis is frequency while the second axis is time.\nWe can display a spectrogram using. `librosa.display.specshow`.","metadata":{}},{"cell_type":"code","source":"X = librosa.stft(x)\nXdb = librosa.amplitude_to_db(abs(X))\nplt.figure(figsize=(14, 5))\nlibrosa.display.specshow(Xdb, sr=sr, x_axis='time', y_axis='hz')\nplt.colorbar()","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:07.865929Z","iopub.execute_input":"2023-03-20T03:09:07.866372Z","iopub.status.idle":"2023-03-20T03:09:10.299047Z","shell.execute_reply.started":"2023-03-20T03:09:07.866340Z","shell.execute_reply":"2023-03-20T03:09:10.297682Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The vertical axis shows frequencies (from 0 to 10kHz), and the horizontal axis shows the time of the clip. Since we see that all action is taking place at the bottom of the spectrum, we can convert the frequency axis to a logarithmic one.","metadata":{}},{"cell_type":"code","source":"librosa.display.specshow(Xdb, sr=sr, x_axis='time', y_axis='log')\nplt.colorbar()","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:10.300558Z","iopub.execute_input":"2023-03-20T03:09:10.301123Z","iopub.status.idle":"2023-03-20T03:09:12.847087Z","shell.execute_reply.started":"2023-03-20T03:09:10.301078Z","shell.execute_reply":"2023-03-20T03:09:12.846219Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 2.Feature extraction\n\nEvery audio signal consists of many features. However, we must extract the characteristics that are relevant to the problem we are trying to solve. The process of extracting features to use them for analysis is called feature extraction. Let us study about few of the features in detail.","metadata":{}},{"cell_type":"markdown","source":"###  Zero Crossing Rate\nThe zero crossing rate is the rate of sign-changes along a signal, i.e., the rate at which the signal changes from positive to negative or back. \nLet us calculate the zero crossing rate for our example audio clip.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(14, 5))\nlibrosa.display.waveplot(x, sr=sr)","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:12.848241Z","iopub.execute_input":"2023-03-20T03:09:12.848697Z","iopub.status.idle":"2023-03-20T03:09:13.160856Z","shell.execute_reply.started":"2023-03-20T03:09:12.848649Z","shell.execute_reply":"2023-03-20T03:09:13.159726Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Zooming in\nn0 = 9000\nn1 = 9100\nplt.figure(figsize=(14, 5))\nplt.plot(x[n0:n1])\nplt.grid()","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:13.162420Z","iopub.execute_input":"2023-03-20T03:09:13.162796Z","iopub.status.idle":"2023-03-20T03:09:13.323766Z","shell.execute_reply.started":"2023-03-20T03:09:13.162761Z","shell.execute_reply":"2023-03-20T03:09:13.322593Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"zero_crossings = librosa.zero_crossings(x[n0:n1], pad=False)\nprint(sum(zero_crossings))","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:13.325164Z","iopub.execute_input":"2023-03-20T03:09:13.325735Z","iopub.status.idle":"2023-03-20T03:09:13.334141Z","shell.execute_reply.started":"2023-03-20T03:09:13.325695Z","shell.execute_reply":"2023-03-20T03:09:13.332129Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Spectral Centroid\nIt indicates where the ”centre of mass” for a sound is located and is calculated as the weighted mean of the frequencies present in the sound. Consider two songs, one from a blues genre and the other belonging to metal. \n","metadata":{}},{"cell_type":"code","source":"spectral_centroids = librosa.feature.spectral_centroid(x, sr=sr)[0]\nspectral_centroids.shape\n","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:13.336049Z","iopub.execute_input":"2023-03-20T03:09:13.336585Z","iopub.status.idle":"2023-03-20T03:09:13.507611Z","shell.execute_reply.started":"2023-03-20T03:09:13.336535Z","shell.execute_reply":"2023-03-20T03:09:13.506164Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Computing the time variable for visualization\nframes = range(len(spectral_centroids))\nt = librosa.frames_to_time(frames)\n# Normalising the spectral centroid for visualisation\ndef normalize(x, axis=0):\n    return sklearn.preprocessing.minmax_scale(x, axis=axis)\n#Plotting the Spectral Centroid along the waveform\nlibrosa.display.waveplot(x, sr=sr, alpha=0.4)\nplt.plot(t, normalize(spectral_centroids), color='r')","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:13.509998Z","iopub.execute_input":"2023-03-20T03:09:13.510596Z","iopub.status.idle":"2023-03-20T03:09:13.792867Z","shell.execute_reply.started":"2023-03-20T03:09:13.510540Z","shell.execute_reply":"2023-03-20T03:09:13.791839Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Spectral Rolloff\nIt is a measure of the shape of the signal. It represents the frequency below which a specified percentage of the total spectral energy, e.g. 85%, lies.","metadata":{}},{"cell_type":"code","source":"spectral_rolloff = librosa.feature.spectral_rolloff(x+0.01, sr=sr)[0]\nlibrosa.display.waveplot(x, sr=sr, alpha=0.4)\nplt.plot(t, normalize(spectral_rolloff), color='r')","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:13.794316Z","iopub.execute_input":"2023-03-20T03:09:13.794652Z","iopub.status.idle":"2023-03-20T03:09:14.230537Z","shell.execute_reply.started":"2023-03-20T03:09:13.794620Z","shell.execute_reply":"2023-03-20T03:09:14.229066Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Mel-Frequency Cepstral Coefficients\nThe Mel frequency cepstral coefficients (MFCCs) of a signal are a small set of features (usually about 10–20) which concisely describe the overall shape of a spectral envelope. It models the characteristics of the human voice.","metadata":{}},{"cell_type":"code","source":"x, fs = librosa.load('../input/birdsong-recognition/train_audio/nutwoo/XC161356.mp3')\nlibrosa.display.waveplot(x, sr=sr)\n","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:14.232035Z","iopub.execute_input":"2023-03-20T03:09:14.232353Z","iopub.status.idle":"2023-03-20T03:09:15.785427Z","shell.execute_reply.started":"2023-03-20T03:09:14.232323Z","shell.execute_reply":"2023-03-20T03:09:15.784020Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mfccs = librosa.feature.mfcc(x, sr=fs)\nprint(mfccs.shape)\n","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:15.787474Z","iopub.execute_input":"2023-03-20T03:09:15.787925Z","iopub.status.idle":"2023-03-20T03:09:15.837678Z","shell.execute_reply.started":"2023-03-20T03:09:15.787886Z","shell.execute_reply":"2023-03-20T03:09:15.836574Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Displaying  the MFCCs:\nlibrosa.display.specshow(mfccs, sr=sr, x_axis='time')\n","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:15.839057Z","iopub.execute_input":"2023-03-20T03:09:15.839394Z","iopub.status.idle":"2023-03-20T03:09:15.956872Z","shell.execute_reply.started":"2023-03-20T03:09:15.839360Z","shell.execute_reply":"2023-03-20T03:09:15.955202Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here mfcc computed 20 MFCC s over 1062 frames.\nWe can also perform feature scaling such that each coefficient dimension has zero mean and unit variance:","metadata":{}},{"cell_type":"code","source":"mfccs = sklearn.preprocessing.scale(mfccs, axis=1)\nprint(mfccs.mean(axis=1))\nprint(mfccs.var(axis=1))\nlibrosa.display.specshow(mfccs, sr=sr, x_axis='time')","metadata":{"execution":{"iopub.status.busy":"2023-03-20T03:09:15.958617Z","iopub.execute_input":"2023-03-20T03:09:15.959035Z","iopub.status.idle":"2023-03-20T03:09:16.074862Z","shell.execute_reply.started":"2023-03-20T03:09:15.958999Z","shell.execute_reply":"2023-03-20T03:09:16.073886Z"},"trusted":true},"execution_count":null,"outputs":[]}]}