{"cells":[{"metadata":{},"cell_type":"markdown","source":"\n# Cornell BirdSong Recognition","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"\"Meatball","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Contents \n\n* [Basic Exploratory Data analysis](#1) \n\n * [Getting started - import data]()\n * [Missing values]()\n * [Rate of each specie]()\n * [where the recordings come from, the heat map]()\n * [Dates on which samples are collected]()\n * [Length/duration of record]()\n * [Author distribution]()\n \n \n* [Audio Data analysis](#2) \n\n * [Playing audio]()\n * [Visualizing audio in 2D]()\n * [Spectrogram analysis]()\n * [MelSpectrogram analysis]()\n ","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport math\nfrom math import *\n\nfrom plotly.offline import init_notebook_mode, iplot\nimport plotly.graph_objects as go\nfrom ipywidgets import widgets\nfrom ipywidgets import *\n\ninit_notebook_mode(connected=True)\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import cufflinks as cf\ncf.go_offline()\ncf.set_config_file(offline=False, world_readable=True)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"train = pd.read_csv(\"../input/birdsong-recognition/train.csv\",delimiter=\",\",encoding=\"latin\", engine='python')\ntest = pd.read_csv(\"../input/birdsong-recognition/test.csv\",delimiter=\",\",encoding=\"latin\", engine='python')\naudio_summary = pd.read_csv(\"../input/birdsong-recognition/example_test_audio_summary.csv\",delimiter=\",\",encoding=\"latin\", engine='python')\naudio_metadata = pd.read_csv(\"../input/birdsong-recognition/example_test_audio_metadata.csv\",delimiter=\",\",encoding=\"latin\", engine='python')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head(5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Missing values ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(20,10))\nsns.heatmap(train.isna(), cbar=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"missing_rate_train = (train.isna().sum()/train.shape[0]).sort_values()\nnb_missing = train.isna().sum().sort_values()\nprint(f'{\"Variable\" :-<25} {\"missing_rate_train\":-<25} {\"Number of missing values\":-<25}')\nfor n in range(len(missing_rate_train)):\n print(f'{missing_rate_train.index[n] :-<25} {missing_rate_train[n]:-<25} {nb_missing[n]:-<25}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.columns","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(train[\"species\"].unique())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"
\n Observation 1 : We have 264 species !! \n
","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Rate of each specie","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"rate = train[\"species\"].value_counts().sort_values()/264\nprint(f'{\"Target\" :-<40} {\"rate\":-<20}')\nfor n in range(len(rate)):\n print(f'{rate.index[n] :-<40} {rate[n]}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train[\"species\"].value_counts().sort_values().iplot(kind=\"bar\",)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Create a heat map to present of records","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"longitude = pd.to_numeric(train['longitude'], errors='coerce')\nlatitude = pd.to_numeric(train['latitude'], errors='coerce')\ndf = pd.concat([longitude,latitude],axis=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import folium\nfrom folium import Choropleth, Circle, Marker\nfrom folium.plugins import HeatMap, MarkerCluster\nf = folium.Figure(width=1000, height=500)\n\nlongitude = pd.to_numeric(train['longitude'], errors='coerce')\nlatitude = pd.to_numeric(train['latitude'], errors='coerce')\ndf = pd.concat([longitude,latitude],axis=1).dropna()\nm = folium.Map(location=[40, 0], zoom_start=2).add_to(f)\n# Add a heatmap to the base map\nHeatMap(data=df[['latitude', 'longitude']], radius=10).add_to(m)\nm","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Playback used","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train['playback_used'].fillna('Missing',inplace=True)\nlabels=train['playback_used'].value_counts().index\nvalues=train['playback_used'].value_counts().values\nfig = go.Figure(data=[go.Pie(labels=labels, values=values, textinfo='label+percent',\n insidetextorientation='radial'\n )])\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Date of records - Dates on which samples are collected","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train['date'] = train['date'].apply(pd.to_datetime,format='%Y-%m-%d', errors='coerce')\ntrain['date'].value_counts().plot(figsize=(25, 6))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(25, 6))\nax = sns.countplot(train['date'].dt.year.dropna().apply(lambda x : int(x)), palette=\"hls\")\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Length of record","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train[\"length\"].value_counts().sort_values().iplot(kind=\"bar\",)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Author distribution","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train[\"author\"].value_counts().sort_values().iplot(kind=\"bar\",)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Ebird code distribution","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train[\"ebird_code\"].value_counts().sort_values().iplot(kind=\"bar\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Audio information / exploration \nOne of the most popular packages in Python to do music analysis is called librosa. I am inviting you to watch this video on youtube.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from IPython.display import YouTubeVideo\n\nYouTubeVideo('MhOdbtPhbLU', width=800, height=300)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Loading audio file:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import warnings\nwarnings.filterwarnings('ignore')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import librosa\nimport librosa.display\naudio_data = '../input/birdsong-recognition/train_audio/aldfly/XC134874.mp3'\nx , sr = librosa.load(audio_data)\nprint(x.shape, sr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"librosa.load(audio_data,sr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Playing Audio","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import IPython.display as ipd\nipd.Audio(audio_data)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from random import sample \nimport matplotlib.pyplot as plt\nfrom matplotlib import colors as mcolors\ncolors = dict(mcolors.BASE_COLORS, **mcolors.CSS4_COLORS)\nby_hsv = sorted((tuple(mcolors.rgb_to_hsv(mcolors.to_rgba(color)[:3])), name)\n for name, color in colors.items())\ncolorsName = [name for hsv, name in by_hsv]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class AudioProcessing:\n \n def ReadAudio(self,ebird_code,filename):\n audio_file = \"../input/birdsong-recognition/train_audio\" + \"/\" + ebird_code + \"/\" + filename\n x , sr = librosa.load(audio_file)\n return x,sr\n \n def LoadAudio(self,audio_file,sr):\n return librosa.load(audio_file,sr)\n \n def PlayingAudio(self,ebird_code,filename):\n audio_file = \"../input/birdsong-recognition/train_audio\" + \"/\" + ebird_code + \"/\" + filename\n x , sr = librosa.load(audio_file)\n librosa.load(audio_file,sr)\n return ipd.Audio(audio_file)\n \n def DisplayWave(self,ebird_code,filename):\n audio_file = \"../input/birdsong-recognition/train_audio\" + \"/\" + ebird_code + \"/\" + filename\n y, sr = librosa.load(audio_file)\n whale_song, _ = librosa.effects.trim(y)\n plt.figure(figsize=(12, 4))\n librosa.display.waveplot(whale_song, sr=sr)\n plt.show()\n \n def DisplaySpectogram(self,ebird_code,filename):\n audio_file = \"../input/birdsong-recognition/train_audio\" + \"/\" + ebird_code + \"/\" + filename\n x , sr = librosa.load(audio_file)\n Xdb = librosa.amplitude_to_db(abs(librosa.stft(x)))\n plt.figure(figsize=(12, 4))\n librosa.display.specshow(Xdb, sr=sr, x_axis='time', y_axis='hz')\n plt.colorbar()\n plt.show() \n def PlotSampleWave(self,nrows,captions,df):\n ncols=1\n f, ax = plt.subplots(nrows,ncols=ncols,figsize=(ncols*12,nrows*3))\n i = 0\n colors = sample(colorsName,nrows)\n for c in captions:\n samples = df[df['ebird_code']==c]['filename'].sample(ncols).values\n audio_file = \"../input/birdsong-recognition/train_audio\" + \"/\" + c + \"/\" + samples[0]\n y, sr = librosa.load(audio_file)\n whale_song, _ = librosa.effects.trim(y)\n librosa.display.waveplot(whale_song, sr=sr, color = colors[i],ax=ax[i])\n i = i + 1\n for i, name in zip(range(nrows), captions):\n ax[i].set_ylabel(name, fontsize=15)\n plt.tight_layout()\n plt.show()\n \n def PlotSampleSpectrogram(self,nrows,captions,df):\n ncols=1\n f, ax = plt.subplots(nrows,ncols=ncols,figsize=(ncols*12,nrows*3))\n i = 0\n colors = sample(colorsName,nrows)\n for c in captions:\n samples = df[df['ebird_code']==c]['filename'].sample(ncols).values\n audio_file = \"../input/birdsong-recognition/train_audio\" + \"/\" + c + \"/\" + samples[0]\n x, sr = librosa.load(audio_file)\n X = librosa.stft(x)\n Xdb = librosa.amplitude_to_db(abs(X))\n librosa.display.specshow(Xdb, sr=sr, x_axis='time', y_axis='hz',ax=ax[i])\n i = i + 1\n for i, name in zip(range(nrows), captions):\n ax[i].set_ylabel(name, fontsize=15)\n plt.tight_layout()\n plt.show()\n \n def PlotSampleMelSpectrogram(self,nrows,captions,df):\n ncols=1\n f, ax = plt.subplots(nrows,ncols=ncols,figsize=(ncols*12,nrows*3))\n i = 0\n colors = sample(colorsName,nrows)\n for c in captions:\n samples = df[df['ebird_code']==c]['filename'].sample(ncols).values\n audio_file = \"../input/birdsong-recognition/train_audio\" + \"/\" + c + \"/\" + samples[0]\n x, sr = librosa.load(audio_file)\n S = librosa.feature.melspectrogram(x, sr)\n log_S = librosa.power_to_db(S, ref=np.max)\n\n librosa.display.specshow(log_S, sr = sr, hop_length = 500, x_axis = 'time', y_axis = 'log', cmap = 'rainbow', ax=ax[i])\n i = i + 1\n for i, name in zip(range(nrows), captions):\n ax[i].set_ylabel(name, fontsize=15)\n plt.tight_layout()\n plt.show()\n ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 1 - Plot a simple of sound waves ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Comparing wave curve for different birds :","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"N = 5\nebird_code_simple = sample(list(train[\"ebird_code\"].unique()),N)\nAudioProcessing().PlotSampleWave(nrows=N,captions=ebird_code_simple,df=train)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 2 - Spectrogram ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**According to wikipedia :** \n\n- A spectrogram is a visual representation of the spectrum of frequencies of a signal as it varies with time. When applied to an audio signal, spectrograms are sometimes called sonographs, voiceprints, or voicegrams. When the data is represented in a 3D plot they may be called waterfalls.\n\n- Spectrograms are used extensively in the fields of music, linguistics, sonar, radar, speech processing, seismology, and others. Spectrograms of audio can be used to identify spoken words phonetically, and to analyse the various calls of animals.\n\n- A spectrogram can be generated by an optical spectrometer, a bank of band-pass filters, by Fourier transform or by a wavelet transform (in which case it is also known as a scaleogram or scalogram).\n\n- A spectrogram is usually depicted as a heat map, i.e., as an image with the intensity shown by varying the colour or brightness.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Comparing spectrogram curve for different birds :","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"AudioProcessing().PlotSampleSpectrogram(nrows=N,captions=ebird_code_simple,df=train)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3 - MelSpectrogram ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**According to wikipedia :**\n\n- The mel-frequency cepstrum (MFC) is a representation of the short-term power spectrum of a sound, based on a linear cosine transform of a log power spectrum on a nonlinear mel scale of frequency.\n\n- Mel-frequency cepstral coefficients (MFCCs) are coefficients that collectively make up an MFC. They are derived from a type of cepstral representation of the audio clip (a nonlinear \"spectrum-of-a-spectrum\"). The difference between the cepstrum and the mel-frequency cepstrum is that in the MFC, the frequency bands are equally spaced on the mel scale, which approximates the human auditory system's response more closely than the linearly-spaced frequency bands used in the normal cepstrum. This frequency warping can allow for better representation of sound, for example, in audio compression.\n\n- MFCCs are commonly derived as follows:\n 1. Take the Fourier transform of (a windowed excerpt of) a signal.\n 2. Map the powers of the spectrum obtained above onto the mel scale, using triangular overlapping windows.\n 3. Take the logs of the powers at each of the mel frequencies.\n 4. Take the discrete cosine transform of the list of mel log powers, as if it were a signal.\n 5. The MFCCs are the amplitudes of the resulting spectrum.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"AudioProcessing().PlotSampleMelSpectrogram(nrows=N,captions=ebird_code_simple,df=train)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## To be continued ... ","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}