{"nbformat_minor": 1, "cells": [{"metadata": {}, "cell_type": "markdown", "source": ["Here's some **data augmentation methods for audio**. Some of them are inspired from [TF speech recognition tutorial](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/speech_commands)\n", "\n", "I highly recommend everyone new to this competition should take a look at the official baseline method showed above because it can produce 77% acc in public LB, which is higher than 60%+ of the teams (include mine) for now..."]}, {"execution_count": null, "outputs": [], "metadata": {"_cell_guid": "83810bfc-e4ba-4507-98af-eb0bfc9c6b6a", "_uuid": "e086b1ac4255ea78d38031cf14088d8ca2206e1b"}, "cell_type": "code", "source": ["import numpy as np # linear algebra\n", "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n", "import librosa\n", "import matplotlib.pyplot as plt\n", "import os\n", "import cv2\n", "import IPython.display as ipd\n", "\n", "EPS = 1e-8"]}, {"execution_count": null, "outputs": [], "metadata": {"collapsed": true}, "cell_type": "code", "source": ["def get_spectrogram(wav):\n", "    D = librosa.stft(wav, n_fft=480, hop_length=160,\n", "                     win_length=480, window='hamming')\n", "    spect, phase = librosa.magphase(D)\n", "    return spect"]}, {"metadata": {}, "cell_type": "markdown", "source": ["Let's take an example audio first"]}, {"execution_count": null, "outputs": [], "metadata": {}, "cell_type": "code", "source": ["file_path = '../input/train/audio/yes/0a7c2a8d_nohash_0.wav'\n", "wav, sr = librosa.load(file_path, sr=None)\n", "print(wav.shape, wav.max(), wav.min())"]}, {"execution_count": null, "outputs": [], "metadata": {"scrolled": false}, "cell_type": "code", "source": ["ipd.Audio(wav, rate=sr)"]}, {"execution_count": null, "outputs": [], "metadata": {}, "cell_type": "code", "source": ["log_spect = np.log(get_spectrogram(wav))\n", "print('spectrogram shape:', log_spect.shape)\n", "plt.imshow(log_spect, aspect='auto', origin='lower',)\n", "plt.title('spectrogram of origin audio')\n", "plt.show()"]}, {"metadata": {}, "cell_type": "markdown", "source": ["## 1. Time shifting\n", "\n", "slightly shift the starting point of the audio, then pad it to original length."]}, {"execution_count": null, "outputs": [], "metadata": {}, "cell_type": "code", "source": ["start_ = int(np.random.uniform(-4800,4800))\n", "print('time shift: ',start_)\n", "if start_ >= 0:\n", "    wav_time_shift = np.r_[wav[start_:], np.random.uniform(-0.001,0.001, start_)]\n", "else:\n", "    wav_time_shift = np.r_[np.random.uniform(-0.001,0.001, -start_), wav[:start_]]\n", "ipd.Audio(wav_time_shift, rate=sr)"]}, {"execution_count": null, "outputs": [], "metadata": {}, "cell_type": "code", "source": ["log_spect = np.log(get_spectrogram(wav_time_shift)+EPS)\n", "print('spectrogram shape:', log_spect.shape)\n", "plt.imshow(log_spect, aspect='auto', origin='lower',)\n", "plt.title('spectrogram of time shifted audio')\n", "plt.show()"]}, {"metadata": {}, "cell_type": "markdown", "source": ["## 2. Speed tuning\n", "slightly change the speed of the audio, then pad or slice it."]}, {"execution_count": null, "outputs": [], "metadata": {}, "cell_type": "code", "source": ["speed_rate = np.random.uniform(0.7,1.3)\n", "wav_speed_tune = cv2.resize(wav, (1, int(len(wav) * speed_rate))).squeeze()\n", "print('speed rate: %.3f' % speed_rate, '(lower is faster)')\n", "if len(wav_speed_tune) < 16000:\n", "    pad_len = 16000 - len(wav_speed_tune)\n", "    wav_speed_tune = np.r_[np.random.uniform(-0.001,0.001,int(pad_len/2)),\n", "                           wav_speed_tune,\n", "                           np.random.uniform(-0.001,0.001,int(np.ceil(pad_len/2)))]\n", "else: \n", "    cut_len = len(wav_speed_tune) - 16000\n", "    wav_speed_tune = wav_speed_tune[int(cut_len/2):int(cut_len/2)+16000]\n", "print('wav length: ', wav_speed_tune.shape[0])\n", "ipd.Audio(wav_speed_tune, rate=sr)"]}, {"execution_count": null, "outputs": [], "metadata": {}, "cell_type": "code", "source": ["log_spect = np.log(get_spectrogram(wav_speed_tune)+EPS)\n", "print('spectrogram shape:', log_spect.shape)\n", "plt.imshow(log_spect, aspect='auto', origin='lower',)\n", "plt.title('spectrogram of speed tuned audio')\n", "plt.show()"]}, {"metadata": {}, "cell_type": "markdown", "source": ["## 3 & 4. Mix background noise & volume tuning\n", "randomly choose a slice of background noise then mix it with the speech audio, along with randomly volume tuning"]}, {"execution_count": null, "outputs": [], "metadata": {}, "cell_type": "code", "source": ["bg_files = os.listdir('../input/train/audio/_background_noise_/')\n", "bg_files.remove('README.md')\n", "chosen_bg_file = bg_files[np.random.randint(6)]\n", "bg, sr = librosa.load('../input/train/audio/_background_noise_/'+chosen_bg_file, sr=None)\n", "print(chosen_bg_file,'|', bg.shape[0], bg.max(), bg.min())\n", "ipd.Audio(bg, rate=sr) # !! be prepared when playing the noise, bacause it's so ANNOYING !!"]}, {"execution_count": null, "outputs": [], "metadata": {}, "cell_type": "code", "source": ["start_ = np.random.randint(bg.shape[0]-16000)\n", "bg_slice = bg[start_ : start_+16000]\n", "wav_with_bg = wav * np.random.uniform(0.8, 1.2) + \\\n", "              bg_slice * np.random.uniform(0, 0.1)\n", "ipd.Audio(wav_with_bg, rate=sr) "]}, {"execution_count": null, "outputs": [], "metadata": {}, "cell_type": "code", "source": ["log_spect = np.log(get_spectrogram(wav_with_bg)+EPS)\n", "print('spectrogram shape:', log_spect.shape)\n", "plt.imshow(log_spect, aspect='auto', origin='lower',)\n", "plt.title('spectrogram of audio with background noise')\n", "plt.show()"]}, {"metadata": {}, "cell_type": "markdown", "source": ["If you find this helpful, please upvote me, thank you!"]}, {"execution_count": null, "outputs": [], "metadata": {"collapsed": true}, "cell_type": "code", "source": []}], "metadata": {"kernelspec": {"display_name": "Python 3", "name": "python3", "language": "python"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "mimetype": "text/x-python", "nbconvert_exporter": "python", "file_extension": ".py", "name": "python", "pygments_lexer": "ipython3", "version": "3.6.3"}}, "nbformat": 4}