{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Audio Segmentation\n\nMy approach for this competition was to segment each audio clip into small chunks that contain only one or a few bird calls, and train on spectrograms of those. Th competetion is coming to an end and I wanted to share my method of segmentation, hopefully someone finds it useful or can point me to a better implementation :-)"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"from pathlib import Path\n\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport librosa\nimport pandas as pd\nimport IPython.display as ipd","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Load train dataframe and select a row at random"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"BASE_DIR = Path('../input/birdsong-recognition')\ntrain_df = pd.read_csv(BASE_DIR / 'train.csv')\nrandom_row = train_df.sample().squeeze()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Load audio, then plot the waveform and listen to the audio[](http://)"},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_rate = 32000\nfpath = BASE_DIR / 'train_audio' / random_row['ebird_code'] / random_row['filename']\naudio, _ = librosa.core.load(fpath, sr=sample_rate, mono=True)\n\nplt.plot(audio)\nipd.display(ipd.Audio(audio, rate=sample_rate))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here is my Signal-to-Noise-based segmenter, given an audio clip it first tries to estimate the noise level by finding the absmax of small non-overlapping chunks of audio. The smallest absmax is choosen as the noise level. When looking at the waveform of the audio, this can intuitively be though of as the maximum amplitude of a small chunk where there is just noise.\n\nThen we go through the audio signal in longer and overlapping segments, if a segment has an absmax that is significantly larger than the noise level, we keep that segment."},{"metadata":{"trusted":true},"cell_type":"code","source":"class SNRSegmenter(object):\n\n    def __init__(self, sample_rate, segment_len_ms, hop_len_ms, noise_len_ms, call_snr):\n        self.segment_len_samples = int(sample_rate * segment_len_ms / 1000)\n        self.hop_len_samples = int(sample_rate * hop_len_ms / 1000)\n        self.noise_len_samples = int(sample_rate * noise_len_ms / 1000)\n\n        self.call_snr = call_snr\n\n    def _get_noise_level(self, sample):\n        abs_max = []\n        \n        if len(sample) > self.noise_len_samples:\n            idx = 0\n            while idx + self.noise_len_samples < len(sample):\n                abs_max.append(np.max(np.abs(sample[idx:(idx+self.noise_len_samples)])))\n                idx += self.noise_len_samples\n        else:\n            abs_max.append(np.max(np.abs(sample)))\n\n        return min(abs_max)\n\n    def __call__(self, sample):\n        \n        noise_level = self._get_noise_level(sample)\n\n        call_segments = []\n        call_snrs = []\n\n        if len(sample) > self.segment_len_samples:\n            idx = 0\n            while idx + self.segment_len_samples < len(sample):\n                segment = sample[idx:(idx+self.segment_len_samples)]\n                seg_abs_max = np.max(np.abs(segment))\n                if seg_abs_max / noise_level > self.call_snr:\n                    call_segments.append(segment)\n                    call_snrs.append(seg_abs_max / noise_level)\n\n                idx += self.hop_len_samples\n\n        return call_segments, call_snrs","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We specify how long we want the found segments to be, how much overlap we want and how long to want the chunks to be when finding the noise level, then we use the segmenter to get all the relevant segments."},{"metadata":{"trusted":true},"cell_type":"code","source":"segment_len_ms = 2500\nhop_len_ms = 1000\nnoise_len_ms = 500\ncall_snr_thresh = 5\n\nsegmenter = SNRSegmenter(sample_rate, segment_len_ms, hop_len_ms, noise_len_ms, call_snr_thresh)\n\ncalls, call_snrs = segmenter(audio)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"And now we can take a look at some of the found call segments"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.title(f'SNR = {call_snrs[0]}')\nplt.plot(calls[0])\nipd.display(ipd.Audio(calls[0], rate=sample_rate))  ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.title(f'SNR = {call_snrs[5]}')\nplt.plot(calls[5])\nipd.display(ipd.Audio(calls[5], rate=sample_rate))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}