{"cells":[{"metadata":{},"cell_type":"markdown","source":"### The Cornell Lab of Ornithology’s Center for Conservation Bioacoustics (CCB)’s mission is to collect and interpret sounds in nature. The CCB develops innovative conservation technologies to inspire and inform the conservation of wildlife and habitats globally. By partnering with the data science community, the CCB hopes to further its mission and improve the accuracy of soundscape analyses.In this competition, you will identify a wide variety of bird vocalizations in soundscape recordings. Due to the complexity of the recordings, they contain weak labels. There might be anthropogenic sounds (e.g., airplane overflights) or other bird and non-bird (e.g., chipmunk) calls in the background, with a particular labeled bird species in the foreground. Bring your new ideas to build effective detectors and classifiers for analyzing complex soundscape recordings!. To unlock the full potential of these extensive and information-rich sound archives, researchers need good machine listeners to reliably extract as much information as possible to aid data-driven conservation."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\npd.options.display.max_columns = 50\nimport warnings\nwarnings.filterwarnings('ignore')\n# for dirname, _, filenames in os.walk('/kaggle/input'):\n#     for filename in filenames:\n#         print(os.path.join(dirname, filename))\n\n# You can write up to 5GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"%matplotlib inline\nimport plotly.express as px\n\nfrom collections import Counter\n\nimport plotly.graph_objects as go\nfrom plotly.subplots import make_subplots\n\nimport holoviews as hv\nfrom holoviews import opts\n\nimport datashader as ds, datashader.transfer_functions as tf, numpy as np\nfrom datashader import spatial\nimport holoviews.operation.datashader as hd\nfrom holoviews.operation import decimate\n\nfrom functools import partial\nimport datashader as ds\nfrom datashader.utils import export_image\nfrom seaborn import color_palette\nfrom holoviews.element.tiles import StamenTerrain, EsriTerrain\n\nhv.extension('bokeh')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"input_path = \"/kaggle/input/birdsong-recognition/\"\n\nhv_opts = dict(cmap='jet', \n               bgcolor='aqua',\n                fontsize={'xticks':7.7, 'yticks':7},\n                xrotation=90,\n                xaxis='top',\n                yaxis='left',\n                height=2200,\n                width=1300,\n                colorbar=True,\n                tools=['hover'])\n\nhv_bar = dict(fontsize={'xticks':7.7, 'yticks':7},\n#               xrotation=90,\n                xaxis='top',\n                yaxis='left',\n                height=2100,\n                width=800,\n                show_grid=True,\n                invert_axes=True,\n                tools=['hover'])\n\nhv_subplot = dict(fontsize={'xticks':7.7, 'yticks':7},\n#               xrotation=90,\n#                 xaxis='top',\n                yaxis='left',\n                height=300,\n                width=1100,\n                show_grid=True,\n                  shared_axes=False,\n#                 invert_axes=True,\n                tools=['hover'])\n\nhv_spectra = dict(height=250,\n                  width=550,\n                  show_grid=True,\n                  xaxis=None,\n                  yaxis=None,\n                  tools=['hover'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Audio Files Check"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"#check audio files per bird-type\naudio_path = os.path.join(input_path, \"train_audio/\")\naudio_dist = {}\nfor bird_type in os.listdir(audio_path):\n    len_audio = len(os.listdir(audio_path + os.sep + f\"{bird_type}\"))\n    audio_dist[bird_type] = len_audio\n\naudio_df = pd.DataFrame.from_dict(audio_dist, orient='index', \\\n                                  columns=['Audio_Count']).reset_index(drop=False).rename(columns={'index':'Bird_Type'})\n\n\naudio_df.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"hv.Bars(audio_df.sort_values(by='Audio_Count', ascending=False)).opts(**hv_bar, color='lightpink',\n                                                                      title='Audio File Distribution For Different Birds.')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_df = pd.read_csv(os.path.join(input_path, 'train.csv'))\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Train Data"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_df.info()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# [Data Shading](https://holoviews.org/index.html) - Using Lat & Long Information"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"lat_long_df = train_df[['longitude', 'latitude', 'country', 'species', 'ebird_code', 'duration', 'elevation']]\nlat_long_df.replace('Not specified', np.NaN, inplace=True)\nlat_long_df.replace('?', np.NaN, inplace=True)\nlat_long_df.dropna(axis=0, inplace=True)\nlat_long_df['longitude'] = lat_long_df['longitude'].apply(lambda x: float(x))\nlat_long_df['latitude'] = lat_long_df['latitude'].apply(lambda x: float(x))\nlat_long_df[['country', 'species']] = lat_long_df[['country', 'species']].apply(lambda x: x.astype('category'))\n\n#generate Web Mercator format for Latitude and Longitude..\nfrom datashader.utils import lnglat_to_meters as webm\nlat_long_webm = list(lat_long_df[['longitude', 'latitude']].apply(lambda x: webm(*x), axis=1).values)\nlat_long_df.loc[:, 'long_wemr'] = [i[0] for i in lat_long_webm]\nlat_long_df.loc[:, 'lat_wemr'] = [i[1] for i in lat_long_webm]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"decimate.max_samples=10\nx_range,y_range = (-19230442.03453801,  19831389.17363642), (-6933173.79129572, 15142823.60169782)\n\nplot_width  = int(1300)\nplot_height = int(800)\n\nunique_values = lat_long_df['ebird_code'].unique()\ncolors = ['#%02x%02x%02x' % (a, b, c) for a,b,c in np.round(255*np.array(color_palette('plasma',n_colors=len(unique_values)))).astype(int)]\ncolor_key = {val:color for val,color in zip(unique_values,colors)}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"tiles = StamenTerrain().redim.range(x=tuple(x_range), y=tuple(y_range))\nlat_longs = hv.Points(lat_long_df, ['long_wemr', 'lat_wemr']).opts(size=5, alpha=0.7)\n\nshade = hd.datashade(lat_longs,\n                     aggregator=ds.count_cat('ebird_code'),\n                     color_key=color_key)\n\ntiles * hd.dynspread(shade).opts(width=plot_width,\n                                  height=plot_height,\n                                  xaxis=None, yaxis=None)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def create_image(df, country_name, title=None, w=plot_width, h=plot_height, annotate=True):\n    \n    country_lat_long = lat_long_df[lat_long_df['country'] == country_name][['long_wemr', 'lat_wemr', 'ebird_code']]\n    country_lat_long.reset_index(drop=True,inplace=True)\n    country_species = country_lat_long.pop('ebird_code')\n    \n    (long_min, lat_min), (long_max, lat_max) = country_lat_long.min(), country_lat_long.max()\n    \n    longitude_range, latitude_range = (long_min, long_max), (lat_min, lat_max)\n    x_range, y_range = longitude_range, latitude_range\n    \n    country_lat_long.loc[:, 'ebird_code'] = country_species.values\n\n    tiles = EsriTerrain().redim.range(x=tuple(x_range), y=tuple(y_range))\n    \n    lat_longs = hv.Points(country_lat_long, ['long_wemr', 'lat_wemr']).opts(size=25, alpha=0.9)\n    shade = hd.datashade(lat_longs,\n                         aggregator=ds.count_cat('ebird_code'),\n                         color_key=color_key)\n    if annotate:\n        labels = hv.Labels(country_lat_long, ['long_wemr', 'lat_wemr'], 'ebird_code').opts(opts.Labels(text_color='ebird_code',\n                                                                                                        padding=5.5, \n                                                                                                        fontsize=1,\n                                                                                                        text_alpha=0.4))\n        layout = tiles * hd.dynspread(shade).opts(width=w,title=title,\n                                                  fontsize=13,\n                                                height=h,\n                                                xaxis=None,\n                                                yaxis=None) * decimate(labels)\n        return layout\n    \n    else:\n        layout = tiles * hd.dynspread(shade).opts(width=w,title=title,\n                                                height=h,\n                                                xaxis=None,\n                                                yaxis=None)\n        return layout","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Some Variable Distributions"},{"metadata":{},"cell_type":"markdown","source":"# Top 5 Countries"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"top_5 = lat_long_df['country'].value_counts()[:5].index.to_list()\ncountry_layout = []\n\nfor country in top_5:\n    country_layout.append(create_image(lat_long_df, str(country), title=str(country), w=700, h=500, annotate=True))\n    \nlayout = hv.Layout(country_layout).cols(2)\n\ndisplay(layout)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Unique Values Check"},{"metadata":{},"cell_type":"markdown","source":"### Categorical Types"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"cat_unique_df = train_df.select_dtypes(include='object').nunique().reset_index().rename(columns={'index':'Column_Name',\n                                                                                 0 : 'Unique_values'}).sort_values(by='Unique_values')\nhv.Bars(cat_unique_df).opts(**hv_bar, color='aqua', title='Unique Values For Each Catergorical Variable.')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Float Types"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"int_unique_df = train_df.select_dtypes(include=['int', 'float']).nunique().reset_index().rename(columns={'index':'Column_Name',\n                                                                                                   0 : 'Unique_values'}).sort_values(by='Unique_values')\nhv.Bars(int_unique_df).opts(**hv_subplot,\n                            color='lightgreen',\n#                             height=500,\n                            title='Unique Values For Each Integer/Float Variable.')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"hv.Bars(train_df['species'].value_counts()).opts(**hv_bar, color='orange', title='Distribution of Species.')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"hv.Bars(train_df['ebird_code'].value_counts()).opts(**hv_bar, color='orange', title='Distribution of ebird_code.')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Looks like `ebird_code` and `species` has same Distribution."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"hv.Bars(train_df['rating'].value_counts()).opts(**hv_subplot, color='lightblue', title='Distribution of Ratings.')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"hv.Bars(train_df['sampling_rate'].value_counts()).opts(**hv_subplot, color='lightblue', title='Distribution of Sampling Rate for the Audio Files.')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"hv.Bars(train_df['playback_used'].value_counts()).opts(**hv_subplot, color='lightblue', title='Distribution of Playback Audio.')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"hv.Bars(train_df['number_of_notes'].value_counts()).opts(**hv_subplot, color='lightblue', title='Distribution Of Number Of Notes in Audio.')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"hv.Bars(train_df['playback_used'].value_counts()).opts(**hv_subplot, color='lightblue', title='Distribution Of Playback Used.')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"df_date = train_df.groupby(\"date\")[\"species\"].count().reset_index().rename(columns = {\"species\": \"recordings\"})\ndf_date.date = pd.to_datetime(df_date.date, errors = \"coerce\")\ndf_date[\"weekday\"] = df_date.date.dt.day_name()\ndf_date.dropna(inplace = True)\nper_day_records = df_date.groupby('weekday', as_index=False).sum().sort_values(by='weekday')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"sub_1 = hv.Curve(data=df_date).opts(**hv_subplot, color='darkgrey', title='Yearwise Recordings')\nsub_2 = hv.Bars(data=per_day_records).opts(**hv_subplot, color='grey', title='Daywise Recordings')\nhv.Layout([sub_1, sub_2]).cols(1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"hv.BoxWhisker(train_df, vdims='duration', kdims='species').opts(**hv_bar, title='Distribution of Duration Of Audio \\n wrt. Bird Species.')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"countrywise_species_df = train_df.groupby(['country', 'species'], as_index=False)['ebird_code'].count()\nhv_opts['cmap'] = 'viridis'\nhv.HeatMap(countrywise_species_df).opts(**hv_opts, title='Countrywise Bird Species Distribution.')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Audio Data - Feature Extraction using Librosa[[](http://)](http://)\n\nLet's extract some features using Librosa library from the audio signals"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import librosa\nimport random\n\ndef get_file(n=1, species=5):\n    ran_samples = {}\n    \n    for species in list(audio_dist.keys())[:5]:\n        species_samples = os.listdir(audio_path + os.sep + species)\n        ran_samples[species] = random.sample(species_samples, n).pop()\n    \n    return [audio_path + sp + os.sep + file for sp, file in ran_samples.items()]\n    \n\nsample_files = get_file(n=1,species=5)\nprint(sample_files)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"tempogram_info = {}\nchromagram_info = {}\nspectral_bandwidth_info = {}\ntonnetz_info = {}\nmfcc_info = {}\npoly_info = {}\nspec_contrast_info = {}\nfourier_tempo_info = {}\n\n\nfor file in sample_files:\n    print(file)\n    data, sr = librosa.load(file)\n    \n    chromagram_info[file] = librosa.feature.chroma_stft(data, sr=sr)\n    spectral_bandwidth_info[file] = librosa.feature.spectral_bandwidth(data, sr=sr)\n    tonnetz_info[file] = librosa.feature.tonnetz(data, sr=sr)\n    mfcc_info[file] = librosa.feature.mfcc(data, sr=sr)\n    poly_info[file] = librosa.feature.poly_features(data, win_length=15, sr=sr)\n    spec_contrast_info[file] = librosa.feature.spectral_contrast(data, sr=sr)\n    \n    #declare onset strength with hop length for rythmic features aka tempogram..\n    oenv = librosa.onset.onset_strength(y=data, sr=sr, hop_length=512)\n    fourier_tempo_info[file] = np.abs(librosa.feature.fourier_tempogram(onset_envelope=oenv,\n                                                                        sr=sr,\n                                                                        hop_length=512))\n    tempogram_info[file] = librosa.feature.tempogram(onset_envelope=oenv,\n                                                     sr=sr,\n                                                     hop_length=512)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def plot_features(features_dict, title='Chromagram'):\n    layout = []\n\n    for k,v in features_dict.items():\n        species, files = k.split(\"/\")[-2:]\n        gram = hv.Image(features_dict[k]).opts(**hv_spectra, cmap='plasma',\n                                               title=f\"{species.capitalize()}-{files.capitalize()} || {title}\")\n\n        layout.append(gram)\n    \n    plot = hv.Layout(layout).cols(2)\n\n    return plot","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"# [Chromagram](https://librosa.org/librosa/generated/librosa.feature.chroma_stft.html#librosa.feature.chroma_stft) - Compute a chromagram from a waveform"},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_features(chromagram_info)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# [Tonnetz](https://librosa.org/librosa/generated/librosa.feature.tonnetz.html#librosa.feature.tonnetz) - Computes the tonal centroid features (tonnetz)"},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_features(tonnetz_info, title='Tonnetz - Tonal Centroid.')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# [MFCC](https://librosa.org/librosa/generated/librosa.feature.mfcc.html#librosa.feature.mfcc) - Mel-frequency cepstral coefficients (MFCCs)"},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_features(mfcc_info, title='MFCCs.')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# [Poly-Features](https://librosa.org/librosa/generated/librosa.feature.poly_features.html#librosa.feature.poly_features) - Coefficients of fitting an nth-order polynomial to the columns of a spectrogram."},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_features(poly_info, title='Poly Feats. window size 15')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# [Spectral Contrast](https://librosa.org/librosa/generated/librosa.feature.spectral_contrast.html#librosa.feature.spectral_contrast) - Computes Spectral Contrast"},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_features(spec_contrast_info, title='Spectral Contrast.')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Rhythm Features\n\n## [Tempogram](https://librosa.org/librosa/generated/librosa.feature.tempogram.html#id1) - Computes the Auto-Correlation tempogram"},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_features(tempogram_info, title='Auto-Correlation Tempogram')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## [Fourier Tempogram](https://librosa.org/librosa/generated/librosa.feature.fourier_tempogram.html#librosa.feature.fourier_tempogram) - Computes the Fourier Tempogram"},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_features(fourier_tempo_info, title='Fourier Tempogram.')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Spectrogram"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"#let's use one audio file\n\nsample_audio = sample_files[0]\nsample_audio, rate = librosa.load(sample_audio)\n\n#spectrogram ..\nsample_stft = np.abs(librosa.stft(sample_audio))\n#decompose the spectrogram such that components.dot(activations)..\ncomps, acts = librosa.decompose.decompose(sample_stft, n_components=32)\n\n#reconstructed...\nstft_recons = comps.dot(acts)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"stft_glyph = hv.Raster(librosa.amplitude_to_db(sample_stft,\n                                               ref=np.max)).opts(**hv_subplot,\n                                                                              cmap='plasma',\n                                                                              title=\"Spectrogram\")\n\n#decompose..\ncomps_glyph = hv.Raster(librosa.amplitude_to_db(comps,\n                                                ref=np.max)).opts(**hv_subplot,\n                                                                         cmap='plasma',\n                                                                         title='Components')\nacts_glyph = hv.Image(acts).opts(**hv_subplot,\n                                 cmap='plasma',\n                                 title='Activations')\n\n#reconstruct..\nstft_recons_glyph = hv.Raster(librosa.amplitude_to_db(stft_recons,\n                                                      ref=np.max)).opts(**hv_subplot,\n                                                                                     cmap='plasma',\n                                                                                     title='Reconstructed Spectogram | [coms.dot(actss)]')\n\nhv.Layout(stft_glyph + comps_glyph + acts_glyph + stft_recons_glyph).cols(1) ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}