{"nbformat_minor": 1, "cells": [{"source": ["### Voice activity detection example\n", "\n", "in (https://www.kaggle.com/davids1992/data-visualization-and-investigation)[this kernel] it was proposed to shrink the samples to those parts where speech could be identified (to speed up training) using webrtcvad\n", "\n", "(note that this will not work on kaggle.com as webrtcvad is not available there)"], "cell_type": "markdown", "metadata": {}}, {"source": ["import os\n", "import numpy as np\n", "\n", "%matplotlib inline\n", "import matplotlib.pyplot as plt"], "outputs": [], "cell_type": "code", "metadata": {}, "execution_count": 1}, {"source": ["directory with input data"], "cell_type": "markdown", "metadata": {}}, {"source": ["train_audio_path = \"../input/train/audio\""], "outputs": [], "cell_type": "code", "metadata": {"collapsed": true}, "execution_count": 2}, {"source": ["example input file (from the same kernel mentioned above)"], "cell_type": "markdown", "metadata": {}}, {"source": ["filename = 'yes/0a7c2a8d_nohash_0.wav'"], "outputs": [], "cell_type": "code", "metadata": {"collapsed": true}, "execution_count": 3}, {"source": ["read the sound file"], "cell_type": "markdown", "metadata": {}}, {"source": ["from scipy.io import wavfile\n", "sample_rate, samples = wavfile.read(os.path.join(train_audio_path, filename))"], "outputs": [], "cell_type": "code", "metadata": {}, "execution_count": 4}, {"source": ["use the webrtcvad library to identify segments as speech or not"], "cell_type": "markdown", "metadata": {}}, {"source": ["import webrtcvad"], "outputs": [], "cell_type": "code", "metadata": {}, "execution_count": 5}, {"source": ["vad = webrtcvad.Vad()\n", "\n", "# set aggressiveness from 0 to 3\n", "vad.set_mode(3)"], "outputs": [], "cell_type": "code", "metadata": {"collapsed": true}, "execution_count": 6}, {"source": ["convert samples to raw 16 bit per sample stream needed by webrtcvad"], "cell_type": "markdown", "metadata": {}}, {"source": ["import struct\n", "raw_samples = struct.pack(\"%dh\" % len(samples), *samples)"], "outputs": [], "cell_type": "code", "metadata": {"collapsed": true}, "execution_count": 7}, {"source": ["run the detector on windows of 30 ms \n", "(from https://github.com/wiseman/py-webrtcvad/blob/master/example.py)"], "cell_type": "markdown", "metadata": {}}, {"source": ["window_duration = 0.03 # duration in seconds\n", "\n", "samples_per_window = int(window_duration * sample_rate + 0.5)\n", "\n", "bytes_per_sample = 2"], "outputs": [], "cell_type": "code", "metadata": {"collapsed": true}, "execution_count": 8}, {"source": ["segments = []\n", "\n", "for start in np.arange(0, len(samples), samples_per_window):\n", "    stop = min(start + samples_per_window, len(samples))\n", "    \n", "    is_speech = vad.is_speech(raw_samples[start * bytes_per_sample: stop * bytes_per_sample], \n", "                              sample_rate = sample_rate)\n", "\n", "    segments.append(dict(\n", "       start = start,\n", "       stop = stop,\n", "       is_speech = is_speech))\n", "    \n", "    "], "outputs": [], "cell_type": "code", "metadata": {"collapsed": true}, "execution_count": 9}, {"source": ["plot the range of samples identified as speech in orange"], "cell_type": "markdown", "metadata": {}}, {"source": ["plt.figure(figsize = (10,7))\n", "plt.plot(samples)\n", "\n", "ymax = max(samples)\n", "\n", "# plot segment identifed as speech\n", "for segment in segments:\n", "    if segment['is_speech']:\n", "        plt.plot([ segment['start'], segment['stop'] - 1], [ymax * 1.1, ymax * 1.1], color = 'orange')\n", "\n", "plt.xlabel('sample')\n", "plt.grid()"], "outputs": [], "cell_type": "code", "metadata": {}, "execution_count": 10}, {"source": ["listen to the speech only segments"], "cell_type": "markdown", "metadata": {}}, {"source": ["speech_samples = np.concatenate([ samples[segment['start']:segment['stop']] for segment in segments if segment['is_speech']])\n", "\n", "import IPython.display as ipd\n", "ipd.Audio(speech_samples, rate=sample_rate)"], "outputs": [], "cell_type": "code", "metadata": {}, "execution_count": 11}, {"source": [], "outputs": [], "cell_type": "code", "metadata": {"collapsed": true}, "execution_count": null}], "metadata": {"language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "nbconvert_exporter": "python", "mimetype": "text/x-python", "name": "python", "pygments_lexer": "ipython3", "version": "3.4.5"}, "kernelspec": {"language": "python", "name": "python3", "display_name": "Python 3"}}, "nbformat": 4}