{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"![](https://mir-s3-cdn-cf.behance.net/project_modules/disp/3ddf1b112293337.6011b598b1884.jpg)\n\n[Source](https://www.behance.net/gallery/112293337/KokoSzanel?tracking_source=search_projects_recommended%7Cbirds)","metadata":{}},{"cell_type":"markdown","source":"# We will mainly use two libraries for audio acquisition and playback:\n## Librosa\nIt is a Python module to analyze audio signals in general but geared more towards music. It includes the nuts and bolts to build a MIR(Music information retrieval) system. It has been very well [documented](https://librosa.org/librosa/) along with a lot of examples and tutorials.\n## IPython.display.Audio\nIPython.display.Audio lets you play audio directly in a jupyter notebook.","metadata":{}},{"cell_type":"code","source":"!pip install librosa","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-02-09T11:43:04.969396Z","iopub.execute_input":"2022-02-09T11:43:04.969912Z","iopub.status.idle":"2022-02-09T11:43:13.234922Z","shell.execute_reply.started":"2022-02-09T11:43:04.969823Z","shell.execute_reply":"2022-02-09T11:43:13.233918Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:13.237993Z","iopub.execute_input":"2022-02-09T11:43:13.238320Z","iopub.status.idle":"2022-02-09T11:43:13.242747Z","shell.execute_reply.started":"2022-02-09T11:43:13.238288Z","shell.execute_reply":"2022-02-09T11:43:13.241819Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Loading an audio file:","metadata":{}},{"cell_type":"code","source":"import librosa\naudio_data = '../input/birdclef-2021/train_short_audio/acafly/XC109605.ogg'\nx , sr = librosa.load(audio_data)\nprint(type(x), type(sr))\nprint(x.shape, sr)","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:13.243853Z","iopub.execute_input":"2022-02-09T11:43:13.244114Z","iopub.status.idle":"2022-02-09T11:43:18.619126Z","shell.execute_reply.started":"2022-02-09T11:43:13.244088Z","shell.execute_reply":"2022-02-09T11:43:18.617748Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### This returns an audio time series as a numpy array with a default sampling rate(sr) of 22KHZ mono. We can change this behavior by resampling at 44.1KHz.","metadata":{}},{"cell_type":"code","source":"librosa.load(audio_data, sr=44100)","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:18.620454Z","iopub.execute_input":"2022-02-09T11:43:18.620768Z","iopub.status.idle":"2022-02-09T11:43:21.485341Z","shell.execute_reply.started":"2022-02-09T11:43:18.620713Z","shell.execute_reply":"2022-02-09T11:43:21.484394Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Playing Audio:\n\n#### Using,IPython.display.Audio you can play the audio in your jupyter notebook.","metadata":{}},{"cell_type":"code","source":"import IPython.display as ipd\nipd.Audio(audio_data)","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:21.488769Z","iopub.execute_input":"2022-02-09T11:43:21.489105Z","iopub.status.idle":"2022-02-09T11:43:21.525553Z","shell.execute_reply.started":"2022-02-09T11:43:21.489071Z","shell.execute_reply":"2022-02-09T11:43:21.524401Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Visualizing Audio:\n\n#### We can plot the audio array using librosa.display.waveplot:","metadata":{}},{"cell_type":"code","source":"%matplotlib inline\nimport matplotlib.pyplot as plt\nimport librosa.display\nplt.figure(figsize=(14, 5))\nlibrosa.display.waveplot(x, sr=sr)","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:21.527164Z","iopub.execute_input":"2022-02-09T11:43:21.527711Z","iopub.status.idle":"2022-02-09T11:43:22.073541Z","shell.execute_reply.started":"2022-02-09T11:43:21.527678Z","shell.execute_reply":"2022-02-09T11:43:22.072375Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Spectrogram\n#### A spectrogram is a visual way of representing the signal strength, or “loudness”, of a signal over time at various frequencies present in a particular waveform. Not only can one see whether there is more or less energy at, for example, 2 Hz vs 10 Hz, but one can also see how energy levels vary over time.\n\n#### A spectrogram is usually depicted as a [heat map](https://en.wikipedia.org/wiki/Heat_map), i.e., as an image with the intensity shown by varying the color or brightness.\n\n### We can display a spectrogram using. librosa.display.specshow.","metadata":{}},{"cell_type":"code","source":"X = librosa.stft(x)\nXdb = librosa.amplitude_to_db(abs(X))\nplt.figure(figsize=(14, 5))\nlibrosa.display.specshow(Xdb, sr=sr, x_axis='time', y_axis='hz')\nplt.colorbar()","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:22.075678Z","iopub.execute_input":"2022-02-09T11:43:22.076198Z","iopub.status.idle":"2022-02-09T11:43:24.160743Z","shell.execute_reply.started":"2022-02-09T11:43:22.076151Z","shell.execute_reply":"2022-02-09T11:43:24.159815Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### .stft() converts data into short term Fourier transform. [STFT](https://www.youtube.com/watch?v=g1_wcbGUcDY) converts signals such that we can know the amplitude of the given frequency at a given time. Using STFT we can determine the amplitude of various frequencies playing at a given time of an audio signal. .specshow is used to display a spectrogram.\n\n#### The vertical axis shows frequencies (from 0 to 10kHz), and the horizontal axis shows the time of the clip. Since we see that all action is taking place at the bottom of the spectrum, we can convert the frequency axis to a logarithmic one.","metadata":{}},{"cell_type":"code","source":"librosa.display.specshow(Xdb, sr=sr, x_axis='time', y_axis='log')\nplt.colorbar()","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:24.162097Z","iopub.execute_input":"2022-02-09T11:43:24.162396Z","iopub.status.idle":"2022-02-09T11:43:26.116140Z","shell.execute_reply.started":"2022-02-09T11:43:24.162367Z","shell.execute_reply":"2022-02-09T11:43:26.115057Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Create an Audio Signal:","metadata":{}},{"cell_type":"code","source":"# In new Librosa version, you need to use soundfile\nimport soundfile as sf","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:26.117900Z","iopub.execute_input":"2022-02-09T11:43:26.118312Z","iopub.status.idle":"2022-02-09T11:43:26.124000Z","shell.execute_reply.started":"2022-02-09T11:43:26.118265Z","shell.execute_reply":"2022-02-09T11:43:26.122291Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sr = 22050 # sample rate\nT = 5.0    # seconds\nt = np.linspace(0, T, int(T*sr), endpoint=False) # time variable\nx = 0.5*np.sin(2*np.pi*220*t)# pure sine wave at 220 Hz\n#Playing the audio\nipd.Audio(x, rate=sr) # load a NumPy array\n#Saving the audio\nsf.write('acafly_XC109605.wav', x, sr)","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:26.125968Z","iopub.execute_input":"2022-02-09T11:43:26.126475Z","iopub.status.idle":"2022-02-09T11:43:26.158924Z","shell.execute_reply.started":"2022-02-09T11:43:26.126422Z","shell.execute_reply":"2022-02-09T11:43:26.157971Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import sklearn\nspectral_centroids = librosa.feature.spectral_centroid(x, sr=sr)[0]\nspectral_centroids.shape\n# Computing the time variable for visualization\nplt.figure(figsize=(12, 4))\nframes = range(len(spectral_centroids))\nt = librosa.frames_to_time(frames)\n# Normalising the spectral centroid for visualisation\ndef normalize(x, axis=0):\n    return sklearn.preprocessing.minmax_scale(x, axis=axis)\n#Plotting the Spectral Centroid along the waveform\nlibrosa.display.waveplot(x, sr=sr, alpha=0.4)\nplt.plot(t, normalize(spectral_centroids), color='b')","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:26.160217Z","iopub.execute_input":"2022-02-09T11:43:26.160891Z","iopub.status.idle":"2022-02-09T11:43:26.389932Z","shell.execute_reply.started":"2022-02-09T11:43:26.160841Z","shell.execute_reply":"2022-02-09T11:43:26.388842Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### .spectral_centroid will return an array with columns equal to a number of frames present in your sample.\n\n# Spectral Rolloff\n### It is a measure of the shape of the signal. It represents the frequency at which high frequencies decline to 0. To obtain it, we have to calculate the fraction of bins in the power spectrum where 85% of its power is at lower frequencies.\n\n### librosa.feature.spectral_rolloff computes the rolloff frequency for each frame in a signal:","metadata":{}},{"cell_type":"code","source":"spectral_rolloff = librosa.feature.spectral_rolloff(x+0.01, sr=sr)[0]\nplt.figure(figsize=(12, 4))\nlibrosa.display.waveplot(x, sr=sr, alpha=0.4)\nplt.plot(t, normalize(spectral_rolloff), color='r')","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:26.391625Z","iopub.execute_input":"2022-02-09T11:43:26.392078Z","iopub.status.idle":"2022-02-09T11:43:26.619538Z","shell.execute_reply.started":"2022-02-09T11:43:26.392031Z","shell.execute_reply":"2022-02-09T11:43:26.618371Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Spectral Bandwidth\n\n#### The spectral bandwidth is defined as the width of the band of light at one-half the peak maximum (or full width at half maximum [FWHM]) and is represented by the two vertical red lines and λSB on the wavelength axis.","metadata":{}},{"cell_type":"code","source":"spectral_bandwidth_2 = librosa.feature.spectral_bandwidth(x+0.01, sr=sr)[0]\nspectral_bandwidth_3 = librosa.feature.spectral_bandwidth(x+0.01, sr=sr, p=3)[0]\nspectral_bandwidth_4 = librosa.feature.spectral_bandwidth(x+0.01, sr=sr, p=4)[0]\nplt.figure(figsize=(15, 9))\nlibrosa.display.waveplot(x, sr=sr, alpha=0.4)\nplt.plot(t, normalize(spectral_bandwidth_2), color='r')\nplt.plot(t, normalize(spectral_bandwidth_3), color='g')\nplt.plot(t, normalize(spectral_bandwidth_4), color='y')\nplt.legend(('p = 2', 'p = 3', 'p = 4'))","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:26.621486Z","iopub.execute_input":"2022-02-09T11:43:26.621984Z","iopub.status.idle":"2022-02-09T11:43:27.028695Z","shell.execute_reply.started":"2022-02-09T11:43:26.621938Z","shell.execute_reply":"2022-02-09T11:43:27.027778Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x, sr = librosa.load(audio_data)\n#Plot the signal:\nplt.figure(figsize=(14, 5))\nlibrosa.display.waveplot(x, sr=sr)\n# Zooming in\nn0 = 9000\nn1 = 9100\nplt.figure(figsize=(14, 5))\nplt.plot(x[n0:n1])\nplt.grid()","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:27.030059Z","iopub.execute_input":"2022-02-09T11:43:27.030361Z","iopub.status.idle":"2022-02-09T11:43:29.685822Z","shell.execute_reply.started":"2022-02-09T11:43:27.030331Z","shell.execute_reply":"2022-02-09T11:43:29.684794Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## How many zero crossings?","metadata":{}},{"cell_type":"code","source":"zero_crossings = librosa.zero_crossings(x[n0:n1], pad=False)\nprint(sum(zero_crossings))","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:29.687815Z","iopub.execute_input":"2022-02-09T11:43:29.688230Z","iopub.status.idle":"2022-02-09T11:43:29.695570Z","shell.execute_reply.started":"2022-02-09T11:43:29.688186Z","shell.execute_reply":"2022-02-09T11:43:29.694333Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Mel-Frequency Cepstral Coefficients (MFCCs)","metadata":{}},{"cell_type":"code","source":"fs=10\nmfccs = librosa.feature.mfcc(x, sr=fs)\nprint(mfccs.shape)\n(20, 97)\n#Displaying  the MFCCs:\nplt.figure(figsize=(15, 7))\nlibrosa.display.specshow(mfccs, sr=sr, x_axis='time')","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:29.697453Z","iopub.execute_input":"2022-02-09T11:43:29.697887Z","iopub.status.idle":"2022-02-09T11:43:30.030252Z","shell.execute_reply.started":"2022-02-09T11:43:29.697840Z","shell.execute_reply":"2022-02-09T11:43:30.029144Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Chroma feature","metadata":{}},{"cell_type":"code","source":"hop_length=12\nchromagram = librosa.feature.chroma_stft(x, sr=sr, hop_length=hop_length)\nplt.figure(figsize=(15, 5))\nlibrosa.display.specshow(chromagram, x_axis='time', y_axis='chroma', hop_length=hop_length, cmap='coolwarm')","metadata":{"execution":{"iopub.status.busy":"2022-02-09T11:43:30.031467Z","iopub.execute_input":"2022-02-09T11:43:30.031937Z"},"trusted":true},"execution_count":null,"outputs":[]}]}