{"cells":[{"metadata":{},"cell_type":"markdown","source":"<img src=\"https://i.imgur.com/jm6Sn49.png\">\n\n<h1><center>🦉Cornell BirdSong Recognition🦉</center></h1>\n\n# 1. Introduction\n\nFinally, some *cutie cute* competition involving animals, sounds, nature, earth and all that goodness 🌍💚.\n\nThis is a very new different challenge for me, and not just because it's mainly based on *audio files*, but because the rules are a bit different than what I'm used to. When I joined, I had (still have) some very big issues in understanding not only the *rules of submission*, but also the *data* and ... what all means?\n\nSo, as I go along I will try to bring some clear understanding and also point to some fruitful discussions. Ok, here we go! 🦅\n\n\n### Libraries 📚⬇","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os\n\nimport pandas as pd\nimport numpy as np\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport matplotlib.image as mpimg\nfrom matplotlib.offsetbox import AnnotationBbox, OffsetImage\n\n# Map 1 library\nimport plotly.express as px\n\n# Map 2 libraries\nimport descartes\nimport geopandas as gpd\nfrom shapely.geometry import Point, Polygon\n\n# Librosa Libraries\nimport librosa\nimport librosa.display\nimport IPython.display as ipd\n\nimport sklearn\n\nimport warnings\nwarnings.filterwarnings('ignore')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 2. The .csv files 📁\n\n> 📌**Note**:\n* `train.csv` contains information about the audio files available in `train_audio`. It contains 21,375 datapoints in 35 unique columns.\n* `test.csv` contains only 3 observations (the rest are available in the *hidden test set*).\n\n<div class=\"alert alert-block alert-info\">\n<b>Note:</b> The TRAIN data has 1 labeled bird species per recording. However, in nature usually you can hear tens (even hundreds) of birds in one go, so in TEST set we need to predict 0, 1 or multiple species for one recording. Because of this, in TRAIN we have <code>species</code> column - or <code>primary_label</code> - (main bird), <code>secondary label</code> (other birds heard) and <code>background</code> (background noises, other birds etc.)\n</div>\n \n### Discussions💬\n* [Few questions about test data](https://www.kaggle.com/c/birdsong-recognition/discussion/159123)\n* [Confusion about test set](https://www.kaggle.com/c/birdsong-recognition/discussion/158987)","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"# Import data\ntrain_csv = pd.read_csv(\"../input/birdsong-recognition/train.csv\")\ntest_csv = pd.read_csv(\"../input/birdsong-recognition/test.csv\")\n\n# Create some time features\ntrain_csv['year'] = train_csv['date'].apply(lambda x: x.split('-')[0])\ntrain_csv['month'] = train_csv['date'].apply(lambda x: x.split('-')[1])\ntrain_csv['day_of_month'] = train_csv['date'].apply(lambda x: x.split('-')[2])\n\nprint(\"There are {:,} unique bird species in the dataset.\".format(len(train_csv['species'].unique())))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### TEST.csv - let's take a look here as well before going further\n\n> 📌**Note**:\n* only 3 rows available (rest are in the hidden set)\n* `site`: there are 3 sites in total, with first 2 having labeles every 5 seconds, while site_3 has labels at file level.\n* `row_id`: this is the unique ID that will be used for the submission\n* `seconds`: how long the clip is\n* `audio_id`: `row_id` without site\n\n*PS: \"nocall\" can be also one of the labels (hearing no bird).*","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Inspect text_csv before checking train data\ntest_csv","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 2.1 Time of the Recording ⏰\n\n> 📌**Note**: Majority of the data was registered between 2013 and 2019, during Spring and Summer months (`00` is for the dates 0000-00-00, which are unknown).","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"bird = mpimg.imread('../input/birdcall-recognition-data/pink bird.jpg')\nimagebox = OffsetImage(bird, zoom=0.5)\nxy = (0.5, 0.7)\nab = AnnotationBbox(imagebox, xy, frameon=False, pad=1, xybox=(6.5, 2000))\n\nplt.figure(figsize=(16, 6))\nax = sns.countplot(train_csv['year'], palette=\"hls\")\nax.add_artist(ab)\n\nplt.title(\"Audio Files Registration per Year Made\", fontsize=16)\nplt.xticks(rotation=90, fontsize=13)\nplt.yticks(fontsize=13)\nplt.ylabel(\"Frequency\", fontsize=14)\nplt.xlabel(\"\");","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"bird = mpimg.imread('../input/birdcall-recognition-data/green bird.jpg')\nimagebox = OffsetImage(bird, zoom=0.3)\nxy = (0.5, 0.7)\nab = AnnotationBbox(imagebox, xy, frameon=False, pad=1, xybox=(11, 3000))\n\nplt.figure(figsize=(16, 6))\nax = sns.countplot(train_csv['month'], palette=\"hls\")\nax.add_artist(ab)\n\nplt.title(\"Audio Files Registration per Month Made\", fontsize=16)\nplt.xticks(fontsize=13)\nplt.yticks(fontsize=13)\nplt.ylabel(\"Frequency\", fontsize=14)\nplt.xlabel(\"\");","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 2.2 The Songs 🎼\n\n> 📌**Note**: Pitch is usually unspecified. This is one of the more *miscellaneous* columns, that we need to be careful how we interpret. Most Song Types are *call, song or flight*.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"bird = mpimg.imread('../input/birdcall-recognition-data/orangebird.jpeg')\nimagebox = OffsetImage(bird, zoom=0.12)\nxy = (0.5, 0.7)\nab = AnnotationBbox(imagebox, xy, frameon=False, pad=1, xybox=(3.9, 8600))\n\nplt.figure(figsize=(16, 6))\nax = sns.countplot(train_csv['pitch'], palette=\"hls\", order = train_csv['pitch'].value_counts().index)\nax.add_artist(ab)\n\nplt.title(\"Pitch (quality of sound - how high/low the tone is)\", fontsize=16)\nplt.xticks(fontsize=13)\nplt.yticks(fontsize=13)\nplt.ylabel(\"Frequency\", fontsize=14)\nplt.xlabel(\"\");","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Type Column**:\n\n> 📌**Note**: This column is a bit messy, as the same description can be found under multiple names. Also, there can be multiple descriptions for multiple sounds (one bird song can mean a different thing from another one in the same recording). Some examples are:\n* **alarm call** is: alarm call | alarm call, call \n* **flight call** is: flight call | call, flight call etc.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create a new variable type by exploding all the values\nadjusted_type = train_csv['type'].apply(lambda x: x.split(',')).reset_index().explode(\"type\")\n\n# Strip of white spaces and convert to lower chars\nadjusted_type = adjusted_type['type'].apply(lambda x: x.strip().lower()).reset_index()\nadjusted_type['type'] = adjusted_type['type'].replace('calls', 'call')\n\n# Create Top 15 list with song types\ntop_15 = list(adjusted_type['type'].value_counts().head(15).reset_index()['index'])\ndata = adjusted_type[adjusted_type['type'].isin(top_15)]\n\n# === PLOT ===\nbird = mpimg.imread('../input/birdcall-recognition-data/Eastern Meadowlark.jpg')\nimagebox = OffsetImage(bird, zoom=0.43)\nxy = (0.5, 0.7)\nab = AnnotationBbox(imagebox, xy, frameon=False, pad=1, xybox=(12.4, 5700))\n\nplt.figure(figsize=(16, 6))\nax = sns.countplot(data['type'], palette=\"hls\", order = data['type'].value_counts().index)\nax.add_artist(ab)\n\nplt.title(\"Top 15 Song Types\", fontsize=16)\nplt.ylabel(\"Frequency\", fontsize=14)\nplt.yticks(fontsize=13)\nplt.xticks(rotation=45, fontsize=13)\nplt.xlabel(\"\");","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 2.3 Where is the bird? 📸🔭\n\n> 📌**Note**: In most recordings the birds were seen, usually at an altitude between 0m and 10m.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Top 15 most common elevations\ntop_15 = list(train_csv['elevation'].value_counts().head(15).reset_index()['index'])\ndata = train_csv[train_csv['elevation'].isin(top_15)]\n\n# === PLOT ===\nbird = mpimg.imread('../input/birdcall-recognition-data/blue bird.jpg')\nimagebox = OffsetImage(bird, zoom=0.43)\nxy = (0.5, 0.7)\nab = AnnotationBbox(imagebox, xy, frameon=False, pad=1, xybox=(12.4, 1450))\n\nplt.figure(figsize=(16, 6))\nax = sns.countplot(data['elevation'], palette=\"hls\", order = data['elevation'].value_counts().index)\nax.add_artist(ab)\n\nplt.title(\"Top 15 Elevation Types\", fontsize=16)\nplt.ylabel(\"Frequency\", fontsize=14)\nplt.yticks(fontsize=13)\nplt.xticks(rotation=45, fontsize=13)\nplt.xlabel(\"\");","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create data\ndata = train_csv['bird_seen'].value_counts().reset_index()\n\n# === PLOT ===\nbird = mpimg.imread('../input/birdcall-recognition-data/black bird.jpg')\nimagebox = OffsetImage(bird, zoom=0.22)\nxy = (0.5, 0.7)\nab = AnnotationBbox(imagebox, xy, frameon=False, pad=1, xybox=(15300, 0.95))\n\nplt.figure(figsize=(16, 6))\nax = sns.barplot(x = 'bird_seen', y = 'index', data = data, palette=\"hls\")\nax.add_artist(ab)\n\nplt.title(\"Song was Heard, but was Bird Seen?\", fontsize=16)\nplt.ylabel(\"Frequency\", fontsize=14)\nplt.yticks(fontsize=13)\nplt.xticks(rotation=45, fontsize=13)\nplt.xlabel(\"\");","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 2.4 World View of the Species 🧭🌏\n\n### #1. Countries 🚩\n\n> 📌**Note**: Let's look at **top 15** countries with most recordings. The majority of recordings are located in the US, followed by Canada and Mexico.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Top 15 most common elevations\ntop_15 = list(train_csv['country'].value_counts().head(15).reset_index()['index'])\ndata = train_csv[train_csv['country'].isin(top_15)]\n\n# === PLOT ===\nbird = mpimg.imread('../input/birdcall-recognition-data/fluff ball.jpg')\nimagebox = OffsetImage(bird, zoom=0.6)\nxy = (0.5, 0.7)\nab = AnnotationBbox(imagebox, xy, frameon=False, pad=1, xybox=(12.2, 7000))\n\nplt.figure(figsize=(16, 6))\nax = sns.countplot(data['country'], palette='hls', order = data['country'].value_counts().index)\nax.add_artist(ab)\n\nplt.title(\"Top 15 Countries with most Recordings\", fontsize=16)\nplt.ylabel(\"Frequency\", fontsize=14)\nplt.yticks(fontsize=13)\nplt.xticks(rotation=45, fontsize=13)\nplt.xlabel(\"\");","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### #2. Map View 🧭","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Import gapminder data, where we have country and iso ALPHA codes\ndf = px.data.gapminder().query(\"year==2007\")[[\"country\", \"iso_alpha\"]]\n\n# Merge the tables together (we lose a fiew rows, but not many)\ndata = pd.merge(left=train_csv, right=df, how=\"inner\", on=\"country\")\n\n# Group by country and count how many species can be found in each\ndata = data.groupby(by=[\"country\", \"iso_alpha\"]).count()[\"species\"].reset_index()\n\nfig = px.choropleth(data, locations=\"iso_alpha\", color=\"species\", hover_name=\"country\",\n                    color_continuous_scale=px.colors.sequential.Teal,\n                    title = \"World Map: Recordings per Country\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### #3. Another Map! 😄 Where are our birds? 🦜","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# SHP file\nworld_map = gpd.read_file(\"../input/world-shapefile/world_shapefile.shp\")\n\n# Coordinate reference system\ncrs = {\"init\" : \"epsg:4326\"}\n\n# Lat and Long need to be of type float, not object\ndata = train_csv[train_csv[\"latitude\"] != \"Not specified\"]\ndata[\"latitude\"] = data[\"latitude\"].astype(float)\ndata[\"longitude\"] = data[\"longitude\"].astype(float)\n\n# Create geometry\ngeometry = [Point(xy) for xy in zip(data[\"longitude\"], data[\"latitude\"])]\n\n# Geo Dataframe\ngeo_df = gpd.GeoDataFrame(data, crs=crs, geometry=geometry)\n\n# Create ID for species\nspecies_id = geo_df[\"species\"].value_counts().reset_index()\nspecies_id.insert(0, 'ID', range(0, 0 + len(species_id)))\n\nspecies_id.columns = [\"ID\", \"species\", \"count\"]\n\n# Add ID to geo_df\ngeo_df = pd.merge(geo_df, species_id, how=\"left\", on=\"species\")\n\n# === PLOT ===\nfig, ax = plt.subplots(figsize = (16, 10))\nworld_map.plot(ax=ax, alpha=0.4, color=\"grey\")\n\npalette = iter(sns.hls_palette(len(species_id)))\n\nfor i in range(264):\n    geo_df[geo_df[\"ID\"] == i].plot(ax=ax, markersize=20, color=next(palette), marker=\"o\", label = \"test\");","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 3. The Audio Files🔈🔉🔊\n\n## 3.1 Description\n\n> 📌**Note**: \n* **train_audio**: short recording (majority in mp3 format) of INDIVIDUAL birds.\n* **test_audio**: recordings took in 3 locations:\n    * Site 1 and Site 2: recordings 10 mins long (mp3) that have labeled a bird every 5 seconds. This is meant to mimic the *real life scenario*, when you would usually have more than 1 bird (or no bird) singing.\n    * Site 3: recordings labeled at file level (because it is especially hard to have coders trained to label these kind of files)\n    \n## 3.2 Duration and File Types 📁","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Creating Interval for *duration* variable\ntrain_csv['duration_interval'] = \">500\"\ntrain_csv.loc[train_csv['duration'] <= 100, 'duration_interval'] = \"<=100\"\ntrain_csv.loc[(train_csv['duration'] > 100) & (train_csv['duration'] <= 200), 'duration_interval'] = \"100-200\"\ntrain_csv.loc[(train_csv['duration'] > 200) & (train_csv['duration'] <= 300), 'duration_interval'] = \"200-300\"\ntrain_csv.loc[(train_csv['duration'] > 300) & (train_csv['duration'] <= 400), 'duration_interval'] = \"300-400\"\ntrain_csv.loc[(train_csv['duration'] > 400) & (train_csv['duration'] <= 500), 'duration_interval'] = \"400-500\"\n\nbird = mpimg.imread('../input/birdcall-recognition-data/multicolor bird.jpg')\nimagebox = OffsetImage(bird, zoom=0.4)\nxy = (0.5, 0.7)\nab = AnnotationBbox(imagebox, xy, frameon=False, pad=1, xybox=(4.4, 12000))\n\nplt.figure(figsize=(16, 6))\nax = sns.countplot(train_csv['duration_interval'], palette=\"hls\")\nax.add_artist(ab)\n\nplt.title(\"Distribution of Recordings Duration\", fontsize=16)\nplt.ylabel(\"Frequency\", fontsize=14)\nplt.yticks(fontsize=13)\nplt.xticks(rotation=45, fontsize=13)\nplt.xlabel(\"\");","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def show_values_on_bars(axs, h_v=\"v\", space=0.4):\n    def _show_on_single_plot(ax):\n        if h_v == \"v\":\n            for p in ax.patches:\n                _x = p.get_x() + p.get_width() / 2\n                _y = p.get_y() + p.get_height()\n                value = int(p.get_height())\n                ax.text(_x, _y, value, ha=\"center\") \n        elif h_v == \"h\":\n            for p in ax.patches:\n                _x = p.get_x() + p.get_width() + float(space)\n                _y = p.get_y() + p.get_height()\n                value = int(p.get_width())\n                ax.text(_x, _y, value, ha=\"left\")\n\n    if isinstance(axs, np.ndarray):\n        for idx, ax in np.ndenumerate(axs):\n            _show_on_single_plot(ax)\n    else:\n        _show_on_single_plot(axs)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"bird = mpimg.imread('../input/birdcall-recognition-data/yellow birds.jpg')\nimagebox = OffsetImage(bird, zoom=0.6)\nxy = (0.5, 0.7)\nab = AnnotationBbox(imagebox, xy, frameon=False, pad=1, xybox=(2.7, 12000))\n\nplt.figure(figsize=(16, 6))\nax = sns.countplot(train_csv['file_type'], palette = \"hls\", order = train_csv['file_type'].value_counts().index)\nax.add_artist(ab)\n\nshow_values_on_bars(ax, \"v\", 0)\n\nplt.title(\"Recording File Types\", fontsize=16)\nplt.ylabel(\"Frequency\", fontsize=14)\nplt.yticks(fontsize=13)\nplt.xticks(rotation=45, fontsize=13)\nplt.xlabel(\"\");","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 3.3 Listening to some Recordings\n\n> 📌**Note**: What is sound? [In physics, sound is a vibration that propagates as an acoustic wave, through a transmission medium such as a gas, liquid or solid.](https://en.wikipedia.org/wiki/Sound)\n<img src=\"https://i.imgur.com/jic0QY3.png\" width=400>","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create Full Path so we can access data more easily\nbase_dir = '../input/birdsong-recognition/train_audio/'\ntrain_csv['full_path'] = base_dir + train_csv['ebird_code'] + '/' + train_csv['filename']\n\n# Now let's sample a fiew audio files\namered = train_csv[train_csv['ebird_code'] == \"amered\"].sample(1, random_state = 33)['full_path'].values[0]\ncangoo = train_csv[train_csv['ebird_code'] == \"cangoo\"].sample(1, random_state = 33)['full_path'].values[0]\nhaiwoo = train_csv[train_csv['ebird_code'] == \"haiwoo\"].sample(1, random_state = 33)['full_path'].values[0]\npingro = train_csv[train_csv['ebird_code'] == \"pingro\"].sample(1, random_state = 33)['full_path'].values[0]\nvesspa = train_csv[train_csv['ebird_code'] == \"vesspa\"].sample(1, random_state = 33)['full_path'].values[0]\n\nbird_sample_list = [\"amered\", \"cangoo\", \"haiwoo\", \"pingro\", \"vesspa\"]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Ok, let's hear some songs! 🕊🎶","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Amered\nipd.Audio(amered)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Cangoo\nipd.Audio(cangoo)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Haiwoo\nipd.Audio(haiwoo)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Pingro\nipd.Audio(pingro)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Vesspa\nipd.Audio(vesspa)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 3.4 Extracting Features from Sounds 🔓\n\n> 📌The audio data is composed by:\n1. **Sound**: sequence of vibrations in varying pressure strengths (`y`)\n2. **Sample Rate**: (`sr`) is the number of samples of audio carried per second, measured in Hz or kHz","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Importing 1 file\ny, sr = librosa.load(vesspa)\n\nprint('y:', y, '\\n')\nprint('y shape:', np.shape(y), '\\n')\nprint('Sample Rate (KHz):', sr, '\\n')\n\n# Verify length of the audio\nprint('Check Len of Audio:', np.shape(y)[0]/sr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Trim leading and trailing silence from an audio signal (silence before and after the actual audio)\naudio_file, _ = librosa.effects.trim(y)\n\n# the result is an numpy ndarray\nprint('Audio File:', audio_file, '\\n')\nprint('Audio File shape:', np.shape(audio_file))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Importing the 5 files\ny_amered, sr_amered = librosa.load(amered)\naudio_amered, _ = librosa.effects.trim(y_amered)\n\ny_cangoo, sr_cangoo = librosa.load(cangoo)\naudio_cangoo, _ = librosa.effects.trim(y_cangoo)\n\ny_haiwoo, sr_haiwoo = librosa.load(haiwoo)\naudio_haiwoo, _ = librosa.effects.trim(y_haiwoo)\n\ny_pingro, sr_pingro = librosa.load(pingro)\naudio_pingro, _ = librosa.effects.trim(y_pingro)\n\ny_vesspa, sr_vesspa = librosa.load(vesspa)\naudio_vesspa, _ = librosa.effects.trim(y_vesspa)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### #1. Sound Waves🌊 (2D Representation)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(5, figsize = (16, 9))\nfig.suptitle('Sound Waves', fontsize=16)\n\nlibrosa.display.waveplot(y = audio_amered, sr = sr_amered, color = \"#A300F9\", ax=ax[0])\nlibrosa.display.waveplot(y = audio_cangoo, sr = sr_cangoo, color = \"#4300FF\", ax=ax[1])\nlibrosa.display.waveplot(y = audio_haiwoo, sr = sr_haiwoo, color = \"#009DFF\", ax=ax[2])\nlibrosa.display.waveplot(y = audio_pingro, sr = sr_pingro, color = \"#00FFB0\", ax=ax[3])\nlibrosa.display.waveplot(y = audio_vesspa, sr = sr_vesspa, color = \"#D9FF00\", ax=ax[4]);\n\nfor i, name in zip(range(5), bird_sample_list):\n    ax[i].set_ylabel(name, fontsize=13)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### #2. Fourier Transform 🥁\n\n> 📌**Note**: Function that gets a signal in the time domain as input, and outputs its decomposition into frequencies. Transform both the y-axis (frequency) to log scale, and the “color” axis (amplitude) to Decibels, which is approx. the log scale of amplitudes.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Default FFT window size\nn_fft = 2048 # FFT window size\nhop_length = 512 # number audio of frames between STFT columns (looks like a good default)\n\n# Short-time Fourier transform (STFT)\nD_amered = np.abs(librosa.stft(audio_amered, n_fft = n_fft, hop_length = hop_length))\nD_cangoo = np.abs(librosa.stft(audio_cangoo, n_fft = n_fft, hop_length = hop_length))\nD_haiwoo = np.abs(librosa.stft(audio_haiwoo, n_fft = n_fft, hop_length = hop_length))\nD_pingro = np.abs(librosa.stft(audio_pingro, n_fft = n_fft, hop_length = hop_length))\nD_vesspa = np.abs(librosa.stft(audio_vesspa, n_fft = n_fft, hop_length = hop_length))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Shape of D object:', np.shape(D_amered))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### #3. Spectrogram 🎷\n\n> 📌**Note**: \n* What is a spectrogram? A spectrogram is a visual representation of the spectrum of frequencies of a signal as it varies with time. When applied to an audio signal, spectrograms are sometimes called sonographs, voiceprints, or voicegrams (wiki).\n* Here we convert the frequency axis to a logarithmic one.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Convert an amplitude spectrogram to Decibels-scaled spectrogram.\nDB_amered = librosa.amplitude_to_db(D_amered, ref = np.max)\nDB_cangoo = librosa.amplitude_to_db(D_cangoo, ref = np.max)\nDB_haiwoo = librosa.amplitude_to_db(D_haiwoo, ref = np.max)\nDB_pingro = librosa.amplitude_to_db(D_pingro, ref = np.max)\nDB_vesspa = librosa.amplitude_to_db(D_vesspa, ref = np.max)\n\n# === PLOT ===\nfig, ax = plt.subplots(2, 3, figsize=(16, 9))\nfig.suptitle('Spectrogram', fontsize=16)\nfig.delaxes(ax[1, 2])\n\nlibrosa.display.specshow(DB_amered, sr = sr_amered, hop_length = hop_length, x_axis = 'time', \n                         y_axis = 'log', cmap = 'cool', ax=ax[0, 0])\nlibrosa.display.specshow(DB_cangoo, sr = sr_cangoo, hop_length = hop_length, x_axis = 'time', \n                         y_axis = 'log', cmap = 'cool', ax=ax[0, 1])\nlibrosa.display.specshow(DB_haiwoo, sr = sr_haiwoo, hop_length = hop_length, x_axis = 'time', \n                         y_axis = 'log', cmap = 'cool', ax=ax[0, 2])\nlibrosa.display.specshow(DB_pingro, sr = sr_pingro, hop_length = hop_length, x_axis = 'time', \n                         y_axis = 'log', cmap = 'cool', ax=ax[1, 0])\nlibrosa.display.specshow(DB_vesspa, sr = sr_vesspa, hop_length = hop_length, x_axis = 'time', \n                         y_axis = 'log', cmap = 'cool', ax=ax[1, 1]);\n\nfor i, name in zip(range(0, 2*3), bird_sample_list):\n    x = i // 3\n    y = i % 3\n    ax[x, y].set_title(name, fontsize=13) ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### #4. Mel Spectrogram 🎷\n> 📌**Note**: The Mel Scale, mathematically speaking, is the result of some non-linear transformation of the frequency scale. The Mel Spectrogram is a normal Spectrogram, but with a Mel Scale on the y axis.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create the Mel Spectrograms\nS_amered = librosa.feature.melspectrogram(y_amered, sr=sr_amered)\nS_DB_amered = librosa.amplitude_to_db(S_amered, ref=np.max)\n\nS_cangoo = librosa.feature.melspectrogram(y_cangoo, sr=sr_cangoo)\nS_DB_cangoo = librosa.amplitude_to_db(S_cangoo, ref=np.max)\n\nS_haiwoo = librosa.feature.melspectrogram(y_haiwoo, sr=sr_haiwoo)\nS_DB_haiwoo = librosa.amplitude_to_db(S_haiwoo, ref=np.max)\n\nS_pingro = librosa.feature.melspectrogram(y_pingro, sr=sr_pingro)\nS_DB_pingro = librosa.amplitude_to_db(S_pingro, ref=np.max)\n\nS_vesspa = librosa.feature.melspectrogram(y_vesspa, sr=sr_vesspa)\nS_DB_vesspa = librosa.amplitude_to_db(S_vesspa, ref=np.max)\n\n# === PLOT ====\nfig, ax = plt.subplots(2, 3, figsize=(16, 9))\nfig.suptitle('Mel Spectrogram', fontsize=16)\nfig.delaxes(ax[1, 2])\n\nlibrosa.display.specshow(S_DB_amered, sr = sr_amered, hop_length = hop_length, x_axis = 'time', \n                         y_axis = 'log', cmap = 'rainbow', ax=ax[0, 0])\nlibrosa.display.specshow(S_DB_cangoo, sr = sr_cangoo, hop_length = hop_length, x_axis = 'time', \n                         y_axis = 'log', cmap = 'rainbow', ax=ax[0, 1])\nlibrosa.display.specshow(S_DB_haiwoo, sr = sr_haiwoo, hop_length = hop_length, x_axis = 'time', \n                         y_axis = 'log', cmap = 'rainbow', ax=ax[0, 2])\nlibrosa.display.specshow(S_DB_pingro, sr = sr_pingro, hop_length = hop_length, x_axis = 'time', \n                         y_axis = 'log', cmap = 'rainbow', ax=ax[1, 0])\nlibrosa.display.specshow(S_DB_vesspa, sr = sr_vesspa, hop_length = hop_length, x_axis = 'time', \n                         y_axis = 'log', cmap = 'rainbow', ax=ax[1, 1]);\n\nfor i, name in zip(range(0, 2*3), bird_sample_list):\n    x = i // 3\n    y = i % 3\n    ax[x, y].set_title(name, fontsize=13)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### #5. Zero Crossing Rate 🚷\n\n> 📌**Note**: the rate at which the signal changes from positive to negative or back.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Total zero_crossings in our 1 song\nzero_amered = librosa.zero_crossings(audio_amered, pad=False)\nzero_cangoo = librosa.zero_crossings(audio_cangoo, pad=False)\nzero_haiwoo = librosa.zero_crossings(audio_haiwoo, pad=False)\nzero_pingro = librosa.zero_crossings(audio_pingro, pad=False)\nzero_vesspa = librosa.zero_crossings(audio_vesspa, pad=False)\n\nzero_birds_list = [zero_amered, zero_cangoo, zero_haiwoo, zero_pingro, zero_vesspa]\n\nfor bird, name in zip(zero_birds_list, bird_sample_list):\n    print(\"{} change rate is {:,}\".format(name, sum(bird)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### #6. Harmonics and Perceptrual 🎹\n\n> 📌**Note**: \n* Harmonics are characteristichs that represent the sound *color*\n* Perceptrual shock wave represents the sound *rhythm and emotion*","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"y_harm_haiwoo, y_perc_haiwoo = librosa.effects.hpss(audio_haiwoo)\n\nplt.figure(figsize = (16, 6))\nplt.plot(y_perc_haiwoo, color = '#FFB100')\nplt.plot(y_harm_haiwoo, color = '#A300F9')\nplt.legend((\"Perceptrual\", \"Harmonics\"))\nplt.title(\"Harmonics and Perceptrual : Haiwoo Bird\", fontsize=16);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### #7. Spectral Centroid 🎯\n\n> 📌**Note**: \nIndicates where the ”centre of mass” for a sound is located and is calculated as the weighted mean of the frequencies present in the sound.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Calculate the Spectral Centroids\nspectral_centroids = librosa.feature.spectral_centroid(audio_cangoo, sr=sr)[0]\n\n# Shape is a vector\nprint('Centroids:', spectral_centroids, '\\n')\nprint('Shape of Spectral Centroids:', spectral_centroids.shape, '\\n')\n\n# Computing the time variable for visualization\nframes = range(len(spectral_centroids))\n\n# Converts frame counts to time (seconds)\nt = librosa.frames_to_time(frames)\n\nprint('frames:', frames, '\\n')\nprint('t:', t)\n\n# Function that normalizes the Sound Data\ndef normalize(x, axis=0):\n    return sklearn.preprocessing.minmax_scale(x, axis=axis)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Plotting the Spectral Centroid along the waveform\nplt.figure(figsize = (16, 6))\nlibrosa.display.waveplot(audio_cangoo, sr=sr, alpha=0.4, color = '#A300F9', lw=3)\nplt.plot(t, normalize(spectral_centroids), color='#FFB100', lw=2)\nplt.legend([\"Spectral Centroid\", \"Wave\"])\nplt.title(\"Spectral Centroid: Cangoo Bird\", fontsize=16);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### #8. Chroma Frequencies\n\n> 📌**Note**: Chroma features are an interesting and powerful representation for music audio in which the entire spectrum is projected onto 12 bins representing the 12 distinct semitones (or chromas) of the musical octave.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Increase or decrease hop_length to change how granular you want your data to be\nhop_length = 5000\n\n# Chromogram Vesspa\nchromagram = librosa.feature.chroma_stft(audio_vesspa, sr=sr_vesspa, hop_length=hop_length)\nprint('Chromogram Vesspa shape:', chromagram.shape)\n\nplt.figure(figsize=(16, 6))\nlibrosa.display.specshow(chromagram, x_axis='time', y_axis='chroma', hop_length=hop_length, cmap='twilight')\n\nplt.title(\"Chromogram: Vesspa\", fontsize=16);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### #9. Tempo BPM (beats per minute)🎤\n> 📌**Note**: Dynamic programming beat tracker.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create Tempo BPM variable\ntempo_amered, _ = librosa.beat.beat_track(y_amered, sr = sr_amered)\ntempo_cangoo, _ = librosa.beat.beat_track(y_cangoo, sr = sr_cangoo)\ntempo_haiwoo, _ = librosa.beat.beat_track(y_haiwoo, sr = sr_haiwoo)\ntempo_pingro, _ = librosa.beat.beat_track(y_pingro, sr = sr_pingro)\ntempo_vesspa, _ = librosa.beat.beat_track(y_vesspa, sr = sr_vesspa)\n\ndata = pd.DataFrame({\"Type\": bird_sample_list , \n                     \"BPM\": [tempo_amered, tempo_cangoo, tempo_haiwoo, tempo_pingro, tempo_vesspa] })\n\n# Image\nbird = mpimg.imread('../input/birdcall-recognition-data/violet bird.jpg')\nimagebox = OffsetImage(bird, zoom=0.34)\nxy = (0.5, 0.7)\nab = AnnotationBbox(imagebox, xy, frameon=False, pad=1, xybox=(0.05, 158))\n\n# Plot\nplt.figure(figsize = (16, 6))\nax = sns.barplot(y = data[\"BPM\"], x = data[\"Type\"], palette=\"hls\")\nax.add_artist(ab)\n\nplt.ylabel(\"BPM\", fontsize=14)\nplt.yticks(fontsize=13)\nplt.xticks(fontsize=13)\nplt.xlabel(\"\")\nplt.title(\"BPM for 5 Different Bird Species\", fontsize=16);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### #10. Spectral Rolloff 🥏\n> 📌**Note**: Is a measure of the *shape of the signal*. It represents the frequency below which a specified percentage of the total spectral energy (e.g. 85%) lies.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Spectral RollOff Vector\nspectral_rolloff = librosa.feature.spectral_rolloff(audio_amered, sr=sr_amered)[0]\n\n# Computing the time variable for visualization\nframes = range(len(spectral_rolloff))\n# Converts frame counts to time (seconds)\nt = librosa.frames_to_time(frames)\n\n# The plot\nplt.figure(figsize = (16, 6))\nlibrosa.display.waveplot(audio_amered, sr=sr_amered, alpha=0.4, color = '#A300F9', lw=3)\nplt.plot(t, normalize(spectral_rolloff), color='#FFB100', lw=3)\nplt.legend([\"Spectral Rolloff\", \"Wave\"])\nplt.title(\"Spectral Rolloff: Amered Bird\", fontsize=16);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"[There are many more features that librosa can extract from sound. Check them all here.](https://librosa.org/librosa/0.6.0/feature.html)\n\n# 4. Additional Data ➕➕➕\n\n**[Why stop at 100? thread here](https://www.kaggle.com/c/birdsong-recognition/discussion/159970)**\n* [@Vopani](https://www.kaggle.com/rohanrao) kindly scraped the remaining of the available bird audios from Xeno-Canto site.\n* The data:\n    * doesn't contain already available train audios from competition data\n    * only MP3 format\n    * same license as the original data\n    * no corrupt audio present\n    \n**However, the Competition Hosts replied with:**\n* limiting to 100 audio files had the reason to not overload the memory\n* only 100 top rated audios were used\n* excluded videos that did not alow derivatives\n\n**They also recommend the following:**\n* the upper limit is usually 500 recordings per species\n* how many recordings are needed to train? (maybe even less than 100?)\n\n> 📌**Note**: So, should you use the extended data? Should you stick only with original data? *I have no clue, try multiple ideas and see what performs better*","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Import the .csv files (corresponding with the extended data)\ntrain_extended_A_Z = pd.read_csv(\"../input/xeno-canto-bird-recordings-extended-a-m/train_extended.csv\")\n\n# Create base directory\nbase_dir_A_M = \"../input/xeno-canto-bird-recordings-extended-a-m\"\nbase_dir_N_Z = \"../input/xeno-canto-bird-recordings-extended-n-z\"\n\n# Create Full Path column to the audio files\ntrain_extended_A_Z['full_path'] = base_dir_A_M + train_extended_A_Z['ebird_code'] + '/' + train_extended_A_Z['filename']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Sanity Check: are all the files the same length?","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def count_files_dir(dir_name = \"Default\", pref = \"Def\"):\n    \n    birds_names = list(os.listdir(dir_name + \"/\" + pref))\n    total_len = 0\n\n    for bird in birds_names:\n        total_len += len(os.listdir(dir_name +\"/\" + pref + \"/\" + bird))\n        \n    return total_len","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"A_M = count_files_dir(base_dir_A_M, pref = \"A-M\")\nN_Z = count_files_dir(base_dir_N_Z, pref = \"N-Z\")\n\nprint(\"There are {:,} birds in A-Z .csv file\".format(len(train_extended_A_Z)), \"\\n\" +\n      \"\\n\" +\n      \"and there are {:,} audio recs.\".format(A_M + N_Z))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Work in Progress ... ⏳","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Just a Kind Reminder to always be Mindful and Better 😊\n\nI would like to take this opportunity to end with a personal note. I joined this competition by having in mind nature and the well being of birds (and all life) on this beautiful planet.\n\nThis competition suprizes 264 total unique beautiful bird species, but our **kind Earth has more than 10,000 unique colorful birds** that are singing out there at the moment.\n\nBut we are [losing species every year due to Global Warming, Polution, and whatever else you would like to name](https://en.wikipedia.org/wiki/List_of_recently_extinct_bird_species).\n\nSo, although our input to this community is quite small and there is not much we can do, we can still be mindful with the effect we have every day to this process and try as much as possible to diminuish it. Some ideas are very simple, like:\n* buying clothes when needed, not only for fashion sake\n* using reusable water bottles\n* drinking the morning coffee home and not on-the-go every day\n* keeping the outdoors clean by keeping the garbage in a bag until you find a trash\n* walking, biking, using public transport more than the car\n* etc.\n\nLet's be **mindful and better**!\nThank you for reading this rant. ","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}