{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"* Importing necessary Libraries\n* Data Source \n* 1. Audio Features\n    * Features extraction\n    * Visualization\n* Data preprocessing \n* Datset Investigation\n* 2. VAD\n* 3. Anamoly Detection \n* 4. Frequency components across the words\n\n------------------------------------","metadata":{}},{"cell_type":"markdown","source":"Source for this work \n- Speech representation and data exploration - DAVIDS -  https://www.kaggle.com/davids1992/speech-representation-and-data-exploration?scriptVersionId=1924001\n- voice activity detection example -ANDRE HOLZNER · - https://www.kaggle.com/holzner/voice-activity-detection-example\n- Voice Activity Detection with webrtcVAD|7z archive -ATUL ANAND {JHA} - https://www.kaggle.com/atulanandjha/voice-activity-detection-with-webrtcvad-7z-archive","metadata":{}},{"cell_type":"markdown","source":"### Importing necessary Libraries","metadata":{}},{"cell_type":"code","source":"import os\nfrom os.path import isdir, join\nfrom scipy.io import wavfile\nfrom subprocess import check_output\nfrom pathlib import Path\nimport pandas as pd\n\n\n# Math\nimport numpy as np\nfrom scipy.fftpack import fft\nfrom scipy import signal\nfrom scipy.io import wavfile\nimport librosa\n\nfrom sklearn.decomposition import PCA\n\n# Visualization\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport IPython.display as ipd\nimport librosa.display\n\nimport plotly.offline as py\npy.init_notebook_mode(connected=True)\nimport plotly.graph_objs as go\nimport plotly.tools as tls\nimport pandas as pd\n\n%matplotlib inline\n","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:38:38.754656Z","iopub.execute_input":"2022-01-07T19:38:38.755220Z","iopub.status.idle":"2022-01-07T19:38:43.553133Z","shell.execute_reply.started":"2022-01-07T19:38:38.755154Z","shell.execute_reply":"2022-01-07T19:38:43.552143Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Install Python packages in an internet-enabled notebook","metadata":{}},{"cell_type":"code","source":"!pip install pyunpack\n!pip install patool","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:38:43.554931Z","iopub.execute_input":"2022-01-07T19:38:43.555181Z","iopub.status.idle":"2022-01-07T19:39:02.803458Z","shell.execute_reply.started":"2022-01-07T19:38:43.555138Z","shell.execute_reply":"2022-01-07T19:39:02.802433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":" # Data Source \n \n Unpack .7z file\n","metadata":{}},{"cell_type":"code","source":"from pyunpack import Archive\nimport shutil\nif not os.path.exists('/kaggle/working/train/'):\n    os.makedirs('/kaggle/working/train/')\nArchive('/kaggle/input/train.7z').extractall('/kaggle/working/train/')\n# for dirname, _, filenames in os.walk('/kaggle/working/train/'):\n#     for filename in filenames:\n#         print(os.path.join(dirname, filename))\n","metadata":{"_kg_hide-output":true,"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-01-07T19:39:02.808423Z","iopub.execute_input":"2022-01-07T19:39:02.808708Z","iopub.status.idle":"2022-01-07T19:40:58.776278Z","shell.execute_reply.started":"2022-01-07T19:39:02.808659Z","shell.execute_reply":"2022-01-07T19:40:58.774503Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"after you are finished working with the images you can delete them so that your commit will succeed (max number of files in working directory for a commit = 500)","metadata":{}},{"cell_type":"code","source":"shutil.make_archive('train/', 'zip', 'train')","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:40:58.779219Z","iopub.execute_input":"2022-01-07T19:40:58.779665Z","iopub.status.idle":"2022-01-07T19:42:52.810366Z","shell.execute_reply.started":"2022-01-07T19:40:58.779592Z","shell.execute_reply":"2022-01-07T19:42:52.809369Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# deleting unwanted extracted files to avoid memory overflow (maxlimit files = 500) while commiting.\n!rm -rf kaggle/working/train/*","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:52.813603Z","iopub.execute_input":"2022-01-07T19:42:52.813860Z","iopub.status.idle":"2022-01-07T19:42:53.580408Z","shell.execute_reply.started":"2022-01-07T19:42:52.813818Z","shell.execute_reply":"2022-01-07T19:42:53.579392Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Loading the trainig Input file.\ntrain_audio_path = \"/kaggle/working/train/train/audio\"","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:53.584137Z","iopub.execute_input":"2022-01-07T19:42:53.584429Z","iopub.status.idle":"2022-01-07T19:42:53.588467Z","shell.execute_reply.started":"2022-01-07T19:42:53.584362Z","shell.execute_reply":"2022-01-07T19:42:53.587495Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\"\"\"\n# It is just a checker code to validate the presence of file.\n\nprint(check_output([\"ls\", \"../input/train/audio\"]).decode(\"utf8\"))\nprint(os.listdir(\"../input/train\"))\n\n\"\"\"\n\n\nprint(check_output([\"ls\", \"/kaggle/working/train/train/audio\"]).decode(\"utf8\"))\nprint(os.listdir(\"/kaggle/working/train/train/audio/yes\"))","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-01-07T19:42:53.589799Z","iopub.execute_input":"2022-01-07T19:42:53.590057Z","iopub.status.idle":"2022-01-07T19:42:53.630553Z","shell.execute_reply.started":"2022-01-07T19:42:53.590003Z","shell.execute_reply":"2022-01-07T19:42:53.629315Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Example input file to be used here...\nfilename = '/yes/00f0204f_nohash_0.wav'","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:53.632417Z","iopub.execute_input":"2022-01-07T19:42:53.632944Z","iopub.status.idle":"2022-01-07T19:42:53.637318Z","shell.execute_reply.started":"2022-01-07T19:42:53.632884Z","shell.execute_reply":"2022-01-07T19:42:53.636542Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dirs = [f for f in os.listdir(train_audio_path) if isdir(join(train_audio_path, f))]\ndirs.sort()\nprint('Number of labels: ' + str(len(dirs)))","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:53.638547Z","iopub.execute_input":"2022-01-07T19:42:53.639014Z","iopub.status.idle":"2022-01-07T19:42:53.653599Z","shell.execute_reply.started":"2022-01-07T19:42:53.638972Z","shell.execute_reply":"2022-01-07T19:42:53.652358Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Audio Features\n\n\n## Features extraction\nA generalized feature extraction algorithm for an audio data sample be like that:\n\n1. Resampling\n2. VAD\n3. Maybe padding with 0 to make signals be equal length\n4. Log spectrogram (or MFCC, or PLP)\n5. Features normalization with mean and std\n6. Stacking of a given number of frames to get temporal information\n\n\n sample_rate, samples = wavfile.read(str(train_audio_path) + filename) \n\n The above code line works fine for everything except **Librosa** library MFCC functionality. So, we'll read wave files using librosa only.\n \n Must to read samples in librosa format. Other wise \"librosa\" error:data must be in floating format","metadata":{"trusted":true}},{"cell_type":"code","source":"samples, sample_rate = librosa.load(str(train_audio_path)+filename)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:53.657401Z","iopub.execute_input":"2022-01-07T19:42:53.658088Z","iopub.status.idle":"2022-01-07T19:42:54.527295Z","shell.execute_reply.started":"2022-01-07T19:42:53.658011Z","shell.execute_reply":"2022-01-07T19:42:54.526438Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualization\n\nThere are two theories of a [human hearing - place](https://en.wikipedia.org/wiki/Place_theory_(hearing) (frequency-based) and [temporal](https://en.wikipedia.org/wiki/Temporal_theory_(hearing) In speech recognition, I see two main tendencies - to input spectrogram (frequencies), and more sophisticated features MFCC - Mel-Frequency Cepstral Coefficients, PLP. You rarely work with raw, temporal data.\n","metadata":{}},{"cell_type":"markdown","source":"### 1.1 Spectogram \n\nDefine a function that calculates spectrogram.\n\nNote, that we are taking logarithm of spectrogram values. It will make our plot much more clear, moreover, it is strictly connected to the way people hear. We need to assure that there are no 0 values as input to logarithm.\n","metadata":{}},{"cell_type":"code","source":"def log_specgram(audio, sample_rate, window_size=20,\n                 step_size=10, eps=1e-10):\n    nperseg = int(round(window_size * sample_rate / 1e3))\n    noverlap = int(round(step_size * sample_rate / 1e3))\n    freqs, times, spec = signal.spectrogram(audio,\n                                    fs=sample_rate,\n                                    window='hann',\n                                    nperseg=nperseg,\n                                    noverlap=noverlap,\n                                    detrend=False)\n    return freqs, times, np.log(spec.T.astype(np.float32) + eps)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:54.528973Z","iopub.execute_input":"2022-01-07T19:42:54.529330Z","iopub.status.idle":"2022-01-07T19:42:54.536630Z","shell.execute_reply.started":"2022-01-07T19:42:54.529277Z","shell.execute_reply":"2022-01-07T19:42:54.535798Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Frequencies are in range (0, 8000) according to Nyquist theorem.\n\nLet's plot it:","metadata":{}},{"cell_type":"code","source":"freqs, times, spectrogram = log_specgram(samples, sample_rate)\n\nfig = plt.figure(figsize=(14, 8))\nax1 = fig.add_subplot(211)\nax1.set_title('Raw wave of ' + filename)\nax1.set_ylabel('Amplitude')\nax1.plot(np.linspace(0, sample_rate/len(samples), sample_rate), samples)\n\nax2 = fig.add_subplot(212)\nax2.imshow(spectrogram.T, aspect='auto', origin='lower', \n           extent=[times.min(), times.max(), freqs.min(), freqs.max()])\nax2.set_yticks(freqs[::16])\nax2.set_xticks(times[::16])\nax2.set_title('Spectrogram of ' + filename)\nax2.set_ylabel('Freqs in Hz')\nax2.set_xlabel('Seconds')","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:54.538038Z","iopub.execute_input":"2022-01-07T19:42:54.538351Z","iopub.status.idle":"2022-01-07T19:42:55.205681Z","shell.execute_reply.started":"2022-01-07T19:42:54.538299Z","shell.execute_reply":"2022-01-07T19:42:55.204984Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"normalizing the audio data. Always a good plan if we gonna feed it into NN.","metadata":{}},{"cell_type":"code","source":"mean = np.mean(spectrogram, axis=0)\nstd = np.std(spectrogram, axis=0)\nspectrogram = (spectrogram - mean) / std","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:55.206862Z","iopub.execute_input":"2022-01-07T19:42:55.207235Z","iopub.status.idle":"2022-01-07T19:42:55.211958Z","shell.execute_reply.started":"2022-01-07T19:42:55.207181Z","shell.execute_reply":"2022-01-07T19:42:55.211126Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There is an interesting fact to point out. We have ~160 features for each frame, frequencies are between 0 and 8000. It means, that one feature corresponds to 50 Hz. However, frequency resolution of the ear is 3.6 Hz within the octave of 1000 – 2000 Hz It means, that people are far more precise and can hear much smaller details than those represented by spectrograms like above.","metadata":{}},{"cell_type":"markdown","source":"### MFCC\n\nIf you want to get to know some details about MFCC take a look at this great tutorial. MFCC explained You can see, that it is well prepared to imitate human hearing properties.\n\nYou can calculate Mel power spectrogram and MFCC using for example librosa python package.\n","metadata":{}},{"cell_type":"code","source":"# From this tutorial\n# https://github.com/librosa/librosa/blob/master/examples/LibROSA%20demo.ipynb\nS = librosa.feature.melspectrogram(samples, sr=sample_rate, n_mels=128)\n\n# Convert to log scale (dB). We'll use the peak power (max) as reference.\nlog_S = librosa.power_to_db(S, ref=np.max)\n\nplt.figure(figsize=(12, 4))\nlibrosa.display.specshow(log_S, sr=sample_rate, x_axis='time', y_axis='mel')\nplt.title('Mel power spectrogram ')\nplt.colorbar(format='%+02.0f dB')\nplt.tight_layout()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-07T19:42:55.213464Z","iopub.execute_input":"2022-01-07T19:42:55.213731Z","iopub.status.idle":"2022-01-07T19:42:56.102071Z","shell.execute_reply.started":"2022-01-07T19:42:55.213683Z","shell.execute_reply":"2022-01-07T19:42:56.101198Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Now delta- mfcc","metadata":{}},{"cell_type":"code","source":"mfcc = librosa.feature.mfcc(S=log_S, n_mfcc=13)\n\n# Let's pad on the first and second deltas while we're at it\ndelta2_mfcc = librosa.feature.delta(mfcc, order=2)\n\nplt.figure(figsize=(12, 4))\nlibrosa.display.specshow(delta2_mfcc)\nplt.ylabel('MFCC coeffs')\nplt.xlabel('Time')\nplt.title('MFCC')\nplt.colorbar()\nplt.tight_layout()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-07T19:42:56.103600Z","iopub.execute_input":"2022-01-07T19:42:56.104113Z","iopub.status.idle":"2022-01-07T19:42:56.445397Z","shell.execute_reply.started":"2022-01-07T19:42:56.104027Z","shell.execute_reply":"2022-01-07T19:42:56.444197Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Spectrogram in 3d ","metadata":{}},{"cell_type":"code","source":"# data = [go.Surface(z=spectrogram.T)]\n# layout = go.Layout(\n#     title='Specgtrogram of \"yes\" in 3d',\n#     scene = dict(\n#     yaxis = dict(title='Frequencies', range=freqs),\n#     xaxis = dict(title='Time', range=times),\n#     zaxis = dict(title='Log amplitude'),\n#     ),\n# )\n# fig = go.Figure(data=data, layout=layout)\n# py.iplot(fig)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:56.447842Z","iopub.execute_input":"2022-01-07T19:42:56.448419Z","iopub.status.idle":"2022-01-07T19:42:56.454181Z","shell.execute_reply.started":"2022-01-07T19:42:56.448203Z","shell.execute_reply":"2022-01-07T19:42:56.453134Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In classical systems, MFCC or similar features are taken as the input to the system instead of spectrograms.\n\nHowever, in end-to-end (often neural-network based) systems, the most common input features are probably raw spectrograms, or mel power spectrograms. For example MFCC decorrelates features, but NNs deal with correlated features well. ","metadata":{}},{"cell_type":"markdown","source":"## 2. Data Preprocessing \n\n### Silence Removal\nAlthough the words are short, there is a lot of silence in them. A decent VAD can reduce training size a lot, accelerating training speed significantly. Let's cut a bit of the file from the beginning and from the end. and listen to it again (based on a plot above, we take from 4000 to 13000):","metadata":{}},{"cell_type":"code","source":"## without silence removal\nipd.Audio(samples, rate=sample_rate)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:56.456146Z","iopub.execute_input":"2022-01-07T19:42:56.456935Z","iopub.status.idle":"2022-01-07T19:42:56.504111Z","shell.execute_reply.started":"2022-01-07T19:42:56.456871Z","shell.execute_reply":"2022-01-07T19:42:56.503067Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# With manual silence removal\nsamples_cut = samples[4000:13000]\nipd.Audio(samples_cut, rate=sample_rate)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:56.506135Z","iopub.execute_input":"2022-01-07T19:42:56.512326Z","iopub.status.idle":"2022-01-07T19:42:56.524152Z","shell.execute_reply.started":"2022-01-07T19:42:56.512239Z","shell.execute_reply":"2022-01-07T19:42:56.522997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can agree that the entire word can be heard. It is impossible to cut all the files manually and do this basing on the simple plot. But you can use for example webrtcvad package to have a good VAD.\n\nLet's plot it again, together with guessed alignment of 'y' 'e' 's' graphems","metadata":{}},{"cell_type":"code","source":"freqs, times, spectrogram_cut = log_specgram(samples_cut, sample_rate)\n\nfig = plt.figure(figsize=(14, 8))\nax1 = fig.add_subplot(211)\nax1.set_title('Raw wave of ' + filename)\nax1.set_ylabel('Amplitude')\nax1.plot(samples_cut)\n\nax2 = fig.add_subplot(212)\nax2.set_title('Spectrogram of ' + filename)\nax2.set_ylabel('Frequencies * 0.1')\nax2.set_xlabel('Samples')\nax2.imshow(spectrogram_cut.T, aspect='auto', origin='lower', \n           extent=[times.min(), times.max(), freqs.min(), freqs.max()])\nax2.set_yticks(freqs[::16])\nax2.set_xticks(times[::16])\nax2.text(0.06, 1000, 'Y', fontsize=18)\nax2.text(0.17, 1000, 'E', fontsize=18)\nax2.text(0.36, 1000, 'S', fontsize=18)\n\nxcoords = [0.025, 0.11, 0.23, 0.49]\nfor xc in xcoords:\n    ax1.axvline(x=xc*16000, c='r')\n    ax2.axvline(x=xc, c='r')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-07T19:42:56.526034Z","iopub.execute_input":"2022-01-07T19:42:56.526679Z","iopub.status.idle":"2022-01-07T19:42:57.203152Z","shell.execute_reply.started":"2022-01-07T19:42:56.526622Z","shell.execute_reply":"2022-01-07T19:42:57.202243Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":" ### Resampling - dimensionality reduction\n \n - reduce the dimensionality of our data is to resample recordings.\n - smaller training size.\n \nYou can hear that the recording don't sound very natural, because they are sampled with 16k frequency, and we usually hear much more. \nHowever, the most speech related frequencies are presented in smaller band. That's why you can still understand another person talking to the telephone, where GSM signal is sampled to 8000 Hz.\n\nSummarizing, we could resample our dataset to 8k. We will discard some information that shouldn't be important, and we'll reduce size of the data.\n\n**FFT (Fast Fourier Transform)** ","metadata":{}},{"cell_type":"code","source":"def custom_fft(y, fs):\n    T = 1.0 / fs\n    N = y.shape[0]\n    yf = fft(y)\n    xf = np.linspace(0.0, 1.0/(2.0*T), N//2)\n    vals = 2.0/N * np.abs(yf[0:N//2])  # FFT is simmetrical, so we take just the first half\n    # FFT is also complex, to we take just the real part (abs)\n    return xf, vals","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:57.204892Z","iopub.execute_input":"2022-01-07T19:42:57.205656Z","iopub.status.idle":"2022-01-07T19:42:57.213729Z","shell.execute_reply.started":"2022-01-07T19:42:57.205580Z","shell.execute_reply":"2022-01-07T19:42:57.212818Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's read some recording, resample it, and listen. We can also compare FFT, Notice, that there is almost no information above 4000 Hz in original signal.","metadata":{}},{"cell_type":"code","source":"# filename = '/happy/0b09edd3_nohash_0.wav'\nfilename ='/yes/00f0204f_nohash_0.wav'\nnew_sample_rate = 8000\n\nsample_rate, samples = wavfile.read(str(train_audio_path) + filename)\nresampled = signal.resample(samples, int(new_sample_rate/sample_rate * samples.shape[0]))","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:57.215438Z","iopub.execute_input":"2022-01-07T19:42:57.216079Z","iopub.status.idle":"2022-01-07T19:42:57.230630Z","shell.execute_reply.started":"2022-01-07T19:42:57.215999Z","shell.execute_reply":"2022-01-07T19:42:57.229720Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# without resampling \nipd.Audio(samples, rate=sample_rate)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:57.231944Z","iopub.execute_input":"2022-01-07T19:42:57.232440Z","iopub.status.idle":"2022-01-07T19:42:57.251099Z","shell.execute_reply.started":"2022-01-07T19:42:57.232398Z","shell.execute_reply":"2022-01-07T19:42:57.249393Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# with resampling \nipd.Audio(resampled, rate=new_sample_rate)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:57.252715Z","iopub.execute_input":"2022-01-07T19:42:57.253215Z","iopub.status.idle":"2022-01-07T19:42:57.266709Z","shell.execute_reply.started":"2022-01-07T19:42:57.253006Z","shell.execute_reply":"2022-01-07T19:42:57.265621Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# At original Sampling \n\nxf, vals = custom_fft(samples, sample_rate)\nplt.figure(figsize=(12, 4))\nplt.title('FFT of recording sampled with ' + str(sample_rate) + ' Hz')\nplt.plot(xf, vals)\nplt.xlabel('Frequency')\nplt.grid()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:57.268357Z","iopub.execute_input":"2022-01-07T19:42:57.268853Z","iopub.status.idle":"2022-01-07T19:42:57.577489Z","shell.execute_reply.started":"2022-01-07T19:42:57.268797Z","shell.execute_reply":"2022-01-07T19:42:57.576473Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# After resampling to reduce traning dat size \n\nxf, vals = custom_fft(resampled, new_sample_rate)\nplt.figure(figsize=(12, 4))\nplt.title('FFT of recording sampled with ' + str(new_sample_rate) + ' Hz')\nplt.plot(xf, vals)\nplt.xlabel('Frequency')\nplt.grid()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:57.579409Z","iopub.execute_input":"2022-01-07T19:42:57.580002Z","iopub.status.idle":"2022-01-07T19:42:57.793150Z","shell.execute_reply.started":"2022-01-07T19:42:57.579945Z","shell.execute_reply":"2022-01-07T19:42:57.792250Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data Set Investigation\n\nNumver of Records ","metadata":{}},{"cell_type":"code","source":"dirs = [f for f in os.listdir(train_audio_path) if isdir(join(train_audio_path, f))]\ndirs.sort()\nprint('Number of labels: ' + str(len(dirs)))","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:57.794866Z","iopub.execute_input":"2022-01-07T19:42:57.795481Z","iopub.status.idle":"2022-01-07T19:42:57.803574Z","shell.execute_reply.started":"2022-01-07T19:42:57.795424Z","shell.execute_reply":"2022-01-07T19:42:57.802662Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Calculate\nnumber_of_recordings = []\nfor direct in dirs:\n    waves = [f for f in os.listdir(join(train_audio_path, direct)) if f.endswith('.wav')]\n    number_of_recordings.append(len(waves))\n\n# Plot\ndata = [go.Histogram(x=dirs, y=number_of_recordings)]\ntrace = go.Bar(\n    x=dirs,\n    y=number_of_recordings,\n    marker=dict(color = number_of_recordings, colorscale='dense', showscale=True\n    ),\n)\nlayout = go.Layout(\n    title='Number of recordings in given label',\n    xaxis = dict(title='Words'),\n    yaxis = dict(title='Number of recordings')\n)\npy.iplot(go.Figure(data=[trace], layout=layout))","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:57.805229Z","iopub.execute_input":"2022-01-07T19:42:57.805849Z","iopub.status.idle":"2022-01-07T19:42:59.733724Z","shell.execute_reply.started":"2022-01-07T19:42:57.805791Z","shell.execute_reply":"2022-01-07T19:42:59.732727Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"split the dataset in a way that one speaker doesn't occur in both train and test sets","metadata":{}},{"cell_type":"code","source":"filenames = ['/yes/00f0204f_nohash_0.wav', '/yes/8830e17f_nohash_2.wav']\nfor filename in filenames:\n    sample_rate, samples = wavfile.read(str(train_audio_path) + filename)\n    xf, vals = custom_fft(samples, sample_rate)\n    plt.figure(figsize=(12, 4))\n    plt.title('FFT of speaker ' + filename[4:11])\n    plt.plot(xf, vals)\n    plt.xlabel('Frequency')\n    plt.grid()\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:42:59.735344Z","iopub.execute_input":"2022-01-07T19:42:59.735606Z","iopub.status.idle":"2022-01-07T19:43:00.361697Z","shell.execute_reply.started":"2022-01-07T19:42:59.735559Z","shell.execute_reply":"2022-01-07T19:43:00.360683Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"filenames = ['on/004ae714_nohash_0.wav', 'on/0137b3f4_nohash_0.wav']\n\nprint('Speaker ' + filenames[0][4:11])\n# Female Speaker\nipd.Audio( join(train_audio_path, filenames[0]), \n          rate=8000)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:00.363364Z","iopub.execute_input":"2022-01-07T19:43:00.363903Z","iopub.status.idle":"2022-01-07T19:43:00.376803Z","shell.execute_reply.started":"2022-01-07T19:43:00.363844Z","shell.execute_reply":"2022-01-07T19:43:00.375787Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Speaker ' + filenames[1][4:11])\n# Male Speaker\nipd.Audio(join(train_audio_path, filenames[1]))","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:00.379141Z","iopub.execute_input":"2022-01-07T19:43:00.379978Z","iopub.status.idle":"2022-01-07T19:43:00.392379Z","shell.execute_reply.started":"2022-01-07T19:43:00.379908Z","shell.execute_reply":"2022-01-07T19:43:00.391215Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"filename = '/yes/01bb6a2a_nohash_1.wav'\nsample_rate, samples = wavfile.read(str(train_audio_path) + filename)\nfreqs, times, spectrogram = log_specgram(samples, sample_rate)\n\nplt.figure(figsize=(10, 7))\nplt.title('Spectrogram of ' + filename)\nplt.ylabel('Freqs')\nplt.xlabel('Time')\nplt.imshow(spectrogram.T, aspect='auto', origin='lower', \n           extent=[times.min(), times.max(), freqs.min(), freqs.max()])\nplt.yticks(freqs[::16])\nplt.xticks(times[::16])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:00.393547Z","iopub.execute_input":"2022-01-07T19:43:00.393966Z","iopub.status.idle":"2022-01-07T19:43:00.704149Z","shell.execute_reply.started":"2022-01-07T19:43:00.393909Z","shell.execute_reply":"2022-01-07T19:43:00.702975Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Recordings length","metadata":{}},{"cell_type":"code","source":"os.listdir(join(train_audio_path, direct))","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:00.712918Z","iopub.execute_input":"2022-01-07T19:43:00.713519Z","iopub.status.idle":"2022-01-07T19:43:00.741770Z","shell.execute_reply.started":"2022-01-07T19:43:00.713328Z","shell.execute_reply":"2022-01-07T19:43:00.741175Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls -la","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:00.743114Z","iopub.execute_input":"2022-01-07T19:43:00.743581Z","iopub.status.idle":"2022-01-07T19:43:01.521431Z","shell.execute_reply.started":"2022-01-07T19:43:00.743538Z","shell.execute_reply":"2022-01-07T19:43:01.520213Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(train_audio_path)\nprint(direct)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:01.523955Z","iopub.execute_input":"2022-01-07T19:43:01.524279Z","iopub.status.idle":"2022-01-07T19:43:01.531239Z","shell.execute_reply.started":"2022-01-07T19:43:01.524221Z","shell.execute_reply":"2022-01-07T19:43:01.529748Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"all the files have 1 second duration:","metadata":{}},{"cell_type":"code","source":"os.listdir(train_audio_path)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:01.533129Z","iopub.execute_input":"2022-01-07T19:43:01.533405Z","iopub.status.idle":"2022-01-07T19:43:01.545729Z","shell.execute_reply.started":"2022-01-07T19:43:01.533356Z","shell.execute_reply":"2022-01-07T19:43:01.544690Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"num_of_shorter = 0\nfor direct in dirs:\n    waves = [f for f in os.listdir(join(train_audio_path, direct)) if f.endswith('.wav')]\n    for wav in waves:\n#         try:\n            sample_rate, samples = wavfile.read(train_audio_path +'/' +direct + '/' + wav)\n            if samples.shape[0] < sample_rate:\n                num_of_shorter += 1\n#         except:\n#             print(\"this gets executed only if there is an error\")\n              \nprint('Number of recordings shorter than 1 second: ' + str(num_of_shorter))\n# example file :'/kaggle/working/train/train/audio_/background_noise_/doing_the_dishes.wav'","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:01.547182Z","iopub.execute_input":"2022-01-07T19:43:01.547416Z","iopub.status.idle":"2022-01-07T19:43:04.744694Z","shell.execute_reply.started":"2022-01-07T19:43:01.547383Z","shell.execute_reply":"2022-01-07T19:43:04.743624Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"###  Mean spectrograms and FFT","metadata":{}},{"cell_type":"code","source":"to_keep = 'yes no up down left right on off stop go'.split()\ndirs = [d for d in dirs if d in to_keep]\n\nprint(dirs)\n\nfor direct in dirs:\n    vals_all = []\n    spec_all = []\n\n    waves = [f for f in os.listdir(join(train_audio_path, direct)) if f.endswith('.wav')]\n    for wav in waves:\n        sample_rate, samples = wavfile.read(train_audio_path +'/' + direct + '/' + wav)\n        if samples.shape[0] != 16000:\n            continue\n        xf, vals = custom_fft(samples, 16000)\n        vals_all.append(vals)\n        freqs, times, spec = log_specgram(samples, 16000)\n        spec_all.append(spec)\n\n    plt.figure(figsize=(14, 4))\n    plt.subplot(121)\n    plt.title('Mean fft of ' + direct)\n    plt.plot(np.mean(np.array(vals_all), axis=0))\n    plt.grid()\n    plt.subplot(122)\n    plt.title('Mean specgram of ' + direct)\n    plt.imshow(np.mean(np.array(spec_all), axis=0).T, aspect='auto', origin='lower', \n               extent=[times.min(), times.max(), freqs.min(), freqs.max()])\n    plt.yticks(freqs[::16])\n    plt.xticks(times[::16])\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:04.746403Z","iopub.execute_input":"2022-01-07T19:43:04.746797Z","iopub.status.idle":"2022-01-07T19:43:44.358809Z","shell.execute_reply.started":"2022-01-07T19:43:04.746730Z","shell.execute_reply":"2022-01-07T19:43:44.357841Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":" **Guassian Miztures modelling** \n \n Kaldi library, that can model words (or smaller parts of words) with GMMs and model temporal dependencies with Hidden Markov Models.","metadata":{}},{"cell_type":"code","source":"def violinplot_frequency(dirs, freq_ind):\n    \"\"\" Plot violinplots for given words (waves in dirs) and frequency freq_ind\n    from all frequencies freqs.\"\"\"\n\n    spec_all = []  # Contain spectrograms\n    ind = 0\n    # taking first 8 words only to keep the plots clean and unclumsy.\n    for direct in dirs[:8]:\n        spec_all.append([])\n\n        waves = [f for f in os.listdir(join(train_audio_path, direct)) if\n                 f.endswith('.wav')]\n        for wav in waves[:100]:\n            sample_rate, samples = wavfile.read(\n                train_audio_path + '/' + direct + '/' + wav)\n            freqs, times, spec = log_specgram(samples, sample_rate)\n            spec_all[ind].extend(spec[:, freq_ind])\n        ind += 1\n\n    # Different lengths = different num of frames. Make number equal\n    minimum = min([len(spec) for spec in spec_all])\n    spec_all = np.array([spec[:minimum] for spec in spec_all])\n\n    plt.figure(figsize=(13,7))\n    plt.title('Frequency ' + str(freqs[freq_ind]) + ' Hz')\n    plt.ylabel('Amount of frequency in a word')\n    plt.xlabel('Words')\n    sns.violinplot(data=pd.DataFrame(spec_all.T, columns=dirs[:8]))\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:44.360537Z","iopub.execute_input":"2022-01-07T19:43:44.361182Z","iopub.status.idle":"2022-01-07T19:43:44.376097Z","shell.execute_reply.started":"2022-01-07T19:43:44.361120Z","shell.execute_reply":"2022-01-07T19:43:44.375210Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"violinplot_frequency(dirs, 20)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:44.377843Z","iopub.execute_input":"2022-01-07T19:43:44.378606Z","iopub.status.idle":"2022-01-07T19:43:45.842328Z","shell.execute_reply.started":"2022-01-07T19:43:44.378486Z","shell.execute_reply":"2022-01-07T19:43:45.841402Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"## 2. Voice Activity Detection ( VAD )","metadata":{}},{"cell_type":"markdown","source":"### use the webrtcvad library to identify segments as speech or not","metadata":{}},{"cell_type":"code","source":"!pip install webrtcvad","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:45.843708Z","iopub.execute_input":"2022-01-07T19:43:45.844174Z","iopub.status.idle":"2022-01-07T19:43:58.037451Z","shell.execute_reply.started":"2022-01-07T19:43:45.844124Z","shell.execute_reply":"2022-01-07T19:43:58.036152Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import webrtcvad","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:58.039354Z","iopub.execute_input":"2022-01-07T19:43:58.039675Z","iopub.status.idle":"2022-01-07T19:43:58.200823Z","shell.execute_reply.started":"2022-01-07T19:43:58.039614Z","shell.execute_reply":"2022-01-07T19:43:58.199976Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### reading the samples and sample_rate feature again to make them compatible with the webrtcvad library. ( it reads at sample_rate = 16000, 32000, 48000; but we had sample_rate = 22050 with librosa)","metadata":{}},{"cell_type":"code","source":"sample_rate, samples = wavfile.read(str(train_audio_path) + filename)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:58.202190Z","iopub.execute_input":"2022-01-07T19:43:58.202469Z","iopub.status.idle":"2022-01-07T19:43:58.207248Z","shell.execute_reply.started":"2022-01-07T19:43:58.202418Z","shell.execute_reply":"2022-01-07T19:43:58.206363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"vad = webrtcvad.Vad()\n# set aggressiveness from 0 to 3\nvad.set_mode(3)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:58.208545Z","iopub.execute_input":"2022-01-07T19:43:58.208819Z","iopub.status.idle":"2022-01-07T19:43:58.219773Z","shell.execute_reply.started":"2022-01-07T19:43:58.208772Z","shell.execute_reply":"2022-01-07T19:43:58.218975Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"convert samples to raw 16 bit per sample stream needed by webrtcvad( there are other options available too , like 32 )","metadata":{}},{"cell_type":"code","source":"import struct\nraw_samples = struct.pack(\"%dh\" % len(samples), *samples)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:58.220976Z","iopub.execute_input":"2022-01-07T19:43:58.221250Z","iopub.status.idle":"2022-01-07T19:43:58.237307Z","shell.execute_reply.started":"2022-01-07T19:43:58.221206Z","shell.execute_reply":"2022-01-07T19:43:58.236447Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"run the detector on windows of 30 ms (from https://github.com/wiseman/py-webrtcvad/blob/master/example.py)","metadata":{}},{"cell_type":"code","source":"window_duration = 0.03 # duration in seconds\nsamples_per_window = int(window_duration * sample_rate + 0.5)\nbytes_per_sample = 2","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Detect Speech instances in an audio","metadata":{}},{"cell_type":"code","source":"segments = []\n\nfor start in np.arange(0, len(samples), samples_per_window):\n    stop = min(start + samples_per_window, len(samples))\n    \n    is_speech = vad.is_speech(raw_samples[start * bytes_per_sample: stop * bytes_per_sample], \n                              sample_rate = sample_rate)\n\n    segments.append(dict(\n       start = start,\n       stop = stop,\n       is_speech = is_speech))","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"plot segment identifed as speech","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize = (10,7))\nplt.plot(samples)\n\nymax = max(samples)\n\n\nfor segment in segments:\n    if segment['is_speech']:\n        plt.plot([ segment['start'], segment['stop'] - 1], [ymax * 1.1, ymax * 1.1], color = 'orange')\n\nplt.xlabel('sample')\nplt.grid()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":" Listen to the speech only segments","metadata":{}},{"cell_type":"code","source":"speech_samples = np.concatenate([ samples[segment['start']:segment['stop']] for segment in segments if segment['is_speech']])\n\nimport IPython.display as ipd\nipd.Audio(speech_samples, rate=sample_rate)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:58.670789Z","iopub.execute_input":"2022-01-07T19:43:58.671174Z","iopub.status.idle":"2022-01-07T19:43:58.684226Z","shell.execute_reply.started":"2022-01-07T19:43:58.671109Z","shell.execute_reply":"2022-01-07T19:43:58.683272Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Till now we have processed for a single audio of any one word : <span style=\"color : blue;\">YES</span> here.\n\n#### Now, its time to have an overall view on other words also. So, lets visualize frequency components for other words as well.","metadata":{}},{"cell_type":"markdown","source":"# 3. Anomaly detection\n\n lower the dimensionality of the dataset and interactively check for any anomaly. We'll use PCA for dimensionality reduction:","metadata":{}},{"cell_type":"code","source":"fft_all = []\nnames = []\nfor direct in dirs:\n    waves = [f for f in os.listdir(join(train_audio_path, direct)) if f.endswith('.wav')]\n    for wav in waves:\n        sample_rate, samples = wavfile.read(train_audio_path+ '/' + direct + '/' + wav)\n        if samples.shape[0] != sample_rate:\n            samples = np.append(samples, np.zeros((sample_rate - samples.shape[0], )))\n        x, val = custom_fft(samples, sample_rate)\n        fft_all.append(val)\n        names.append(direct + '/' + wav)\n\nfft_all = np.array(fft_all)\n\n# Normalization\nfft_all = (fft_all - np.mean(fft_all, axis=0)) / np.std(fft_all, axis=0)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:50:36.383231Z","iopub.execute_input":"2022-01-07T19:50:36.383642Z","iopub.status.idle":"2022-01-07T19:51:01.979896Z","shell.execute_reply.started":"2022-01-07T19:50:36.383573Z","shell.execute_reply":"2022-01-07T19:51:01.978678Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Dimemsionality reduction\npca = PCA(n_components=3)\nfft_all = pca.fit_transform(fft_all)\n\ndef interactive_3d_plot(data, names):\n    scatt = go.Scatter3d(x=data[:, 0], y=data[:, 1], z=data[:, 2], mode='markers', text=names)\n    data = go.Data([scatt])\n    layout = go.Layout(title=\"Anomaly detection\")\n    figure = go.Figure(data=data, layout=layout)\n    py.iplot(figure)\n    \ninteractive_3d_plot(fft_all, names)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:58.957815Z","iopub.status.idle":"2022-01-07T19:43:58.958307Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Some anomalied listed below","metadata":{}},{"cell_type":"code","source":"print('Recording go/0487ba9b_nohash_0.wav')\nipd.Audio(join(train_audio_path, 'go/0487ba9b_nohash_0.wav'))","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:58.959247Z","iopub.status.idle":"2022-01-07T19:43:58.959705Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Recording yes/e4b02540_nohash_0.wav')\nipd.Audio(join(train_audio_path, 'yes/e4b02540_nohash_0.wav'))","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:58.960559Z","iopub.status.idle":"2022-01-07T19:43:58.961011Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Recording seven/e4b02540_nohash_0.wav')\nipd.Audio(join(train_audio_path, 'seven/b1114e4f_nohash_0.wav'))","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:58.961855Z","iopub.status.idle":"2022-01-07T19:43:58.962436Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 4. Frequency components across the words\n\n### plotting for first 8 words only to avoid clumsy tight plots.","metadata":{}},{"cell_type":"code","source":"violinplot_frequency(dirs, 20)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:58.963566Z","iopub.status.idle":"2022-01-07T19:43:58.964145Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"violinplot_frequency(dirs, 50)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:58.965193Z","iopub.status.idle":"2022-01-07T19:43:58.965886Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"violinplot_frequency(dirs, 120)","metadata":{"execution":{"iopub.status.busy":"2022-01-07T19:43:58.967183Z","iopub.status.idle":"2022-01-07T19:43:58.967622Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 5. Testing with WebRTC input and Recorded Wav files ","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 6. trigger word detection(transformer)","metadata":{}},{"cell_type":"markdown","source":"Copyright 2019 The TensorFlow Authors.\n        #@title Licensed under the Apache License, Version 2.0 (the \"License\");\n        # you may not use this file except in compliance with the License.\n        # You may obtain a copy of the License at\n        #\n        # https://www.apache.org/licenses/LICENSE-2.0\n        #\n        # Unless required by applicable law or agreed to in writing, software\n        # distributed under the License is distributed on an \"AS IS\" BASIS,\n        # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n        # See the License for the specific language governing permissions and\n        # limitations under the License.","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"References for this work\n\n- EABDSMD- https://www.kaggle.com/samadi10/trigger-word-detection-transformer ","metadata":{}},{"cell_type":"markdown","source":"--------------------------------------------------\n**Reading material**\n\n* Encoder-decoder: https://arxiv.org/abs/1508.01211\n* RNNs with CTC loss: https://arxiv.org/abs/1412.5567\n* For me, 1 and 2 are a sensible choice for this competition, especially if you do not have background in SR field. They try to be end-to-end solutions. Speech recognition is a really big topic and it would be hard to get to know important things in short time.\n* Classic speech recognition : http://www.ece.ucsb.edu/Faculty/Rabiner/ece259/Reprints/tutorial%20on%20hmm%20and%20applications.pdf\n\n* Kaldi Tutorial for dummies, with a problem similar to this competition in some way.\n\n* Very deep CNN - Large Vocabulary Continuous Speech Recognition Systems (LVCSR). \n","metadata":{}}]}