{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Data augmentation with audiomentation\n\nHere is example notebook of [audiomentation](https://github.com/iver56/audiomentations), a library for audio data augmentaion.\n\n![](http://)I'm new to this community, so any suggestions for better notebook/results/competition are welceome.\n\nAlso, I haven't used this augmentation for submission at this point. Keen to check if these augmentation works well.\n"},{"metadata":{},"cell_type":"markdown","source":"## Prepare for execution"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"!pip install audiomentations\n!pip install wavio pyloudnorm ffmpeg pydub # install extra dependencies","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Load modules and define some useful functions. \nSome part of code inherit from [ResNet34 More Augmentations+Mixup+TTA (Inference)](https://www.kaggle.com/khoongweihao/resnet34-more-augmentations-mixup-tta-inference) and [All-in-one RFCX baseline for beginners](https://www.kaggle.com/c/rfcx-species-audio-detection)."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Define some helper functions for pretty figures.\nimport csv\nimport librosa\nimport librosa.display\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom IPython.display import Audio, display\n\ndef show_signal(raw_wav, sr, title=None):\n    fig = plt.figure(figsize=(5, 5))\n    ax1 = plt.subplot(2, 1, 1)\n    if title:\n        ax1.set_title(title)\n    ax1.plot(np.arange(len(raw_wav))/sr, raw_wav)\n    \n    ax2 = plt.subplot(2, 1, 2, sharex=ax1)\n    wav_stft = librosa.amplitude_to_db(np.abs(librosa.stft(raw_wav)), ref=np.max)\n    librosa.display.specshow(wav_stft, sr=sr, x_axis='time', y_axis='mel')\n    \n    return Audio((raw_wav*2**15).astype(np.int16), rate=sr)\n    \n    \ndef compare_signals(wav1, sr1, wav2, sr2, titles=None):\n    fig = plt.figure(figsize=(10, 5))\n    ax1 = plt.subplot(2, 2, 1)\n    if titles:\n        ax1.set_title(titles[0])\n    ax1.plot(np.arange(len(wav1))/sr, wav1)\n    \n    ax2 = plt.subplot(2, 2, 3, sharex=ax1)\n    wav_stft = librosa.amplitude_to_db(np.abs(librosa.stft(wav1)), ref=np.max)\n    librosa.display.specshow(wav_stft, sr=sr, x_axis='time', y_axis='mel')\n    \n    ax3 = plt.subplot(2, 2, 2)\n    if titles:\n        ax3.set_title(titles[0])\n    ax3.plot(np.arange(len(wav2))/sr, wav2)\n    \n    ax4 = plt.subplot(2, 2, 4, sharex=ax3)\n    wav_stft = librosa.amplitude_to_db(np.abs(librosa.stft(wav2)), ref=np.max)\n    librosa.display.specshow(wav_stft, sr=sr, x_axis='time', y_axis='mel')\n    \n    print(titles[0])\n    display(Audio((wav1*2**15).astype(np.int16), rate=sr1))\n    print(titles[1])\n    display(Audio((wav2*2**15).astype(np.int16), rate=sr2))\n    return display()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Load sample data\n\nThe code here inherit from [All-in-one RFCX baseline for beginners](https://www.kaggle.com/c/rfcx-species-audio-detection) with modifications."},{"metadata":{"trusted":true},"cell_type":"code","source":"with open('/kaggle/input/rfcx-species-audio-detection/train_tp.csv') as f:\n    reader = csv.reader(f)\n    data = list(reader)\n    \nwav, sr = librosa.load('/kaggle/input/rfcx-species-audio-detection/train/' + data[10][0] + '.flac', sr=None)\n\nshow_signal(wav, sr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Test augmentations\n\nProbability $p$ is fixed to 1.0, which means the augmentation always (100%) happens. "},{"metadata":{"trusted":true},"cell_type":"code","source":"from audiomentations import *","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Do the official example"},{"metadata":{"trusted":true},"cell_type":"code","source":"SAMPLE_RATE = 16000\n\naugment = Compose([\n    AddGaussianNoise(min_amplitude=0.001, max_amplitude=0.015, p=0.5),\n    TimeStretch(min_rate=0.8, max_rate=1.25, p=0.5),\n    PitchShift(min_semitones=-4, max_semitones=4, p=0.5),\n    Shift(min_fraction=-0.5, max_fraction=0.5, p=0.5),\n])\n\n# Generate 2 seconds of dummy audio for the sake of example\nsamples = np.random.uniform(low=-0.2, high=0.2, size=(32000,)).astype(np.float32)\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=samples, sample_rate=SAMPLE_RATE)\nprint(augmented_samples)\n\ncompare_signals(samples, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### AddBackgroundNoise"},{"metadata":{"trusted":true},"cell_type":"code","source":"# use train data as additive noise\naugment = Compose([\n    AddBackgroundNoise(sounds_path=\"../input/rfcx-species-audio-detection/train/\", \n                       min_snr_in_db=3, \n                       max_snr_in_db=30, \n                       p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### AddGaussianNoise"},{"metadata":{"trusted":true},"cell_type":"code","source":"# use default values.\naugment = Compose([\n    AddGaussianNoise(min_amplitude=0.001, \n                       max_amplitude=0.015, \n                       p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### AddShortNoises"},{"metadata":{"trusted":true},"cell_type":"code","source":"# use train data as noise\naugment = Compose([\n    AddShortNoises(sounds_path=\"../input/rfcx-species-audio-detection/train/\",\n                   min_snr_in_db=0,\n                   max_snr_in_db=24,\n                   min_time_between_sounds=4.0,\n                   max_time_between_sounds=16.0,\n                   burst_probability=0.22,\n                   min_pause_factor_during_burst=0.1,\n                   max_pause_factor_during_burst=1.1,\n                   min_fade_in_time=0.005,\n                   max_fade_in_time=0.08,\n                   min_fade_out_time=0.01,\n                   max_fade_out_time=0.1,\n                   p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### ClippingDistortion"},{"metadata":{"trusted":true},"cell_type":"code","source":"augment = Compose([\n    ClippingDistortion(min_percentile_threshold=0, \n                       max_percentile_threshold=40, \n                       p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### FrequencyMask"},{"metadata":{"trusted":true},"cell_type":"code","source":"augment = Compose([\n    FrequencyMask(min_frequency_band=0.0, \n                  max_frequency_band=0.5, \n                  p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Gain"},{"metadata":{"trusted":true},"cell_type":"code","source":"augment = Compose([\n    Gain(min_gain_in_db=-12, \n         max_gain_in_db=12, \n         p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Mp3Compression"},{"metadata":{"trusted":true},"cell_type":"code","source":"augment = Compose([\n    Mp3Compression(min_bitrate=8, \n                   max_bitrate=64, \n                   backend=\"pydub\",\n                   p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### LoudnessNormalization"},{"metadata":{"trusted":true},"cell_type":"code","source":"augment = Compose([\n    LoudnessNormalization(min_lufs_in_db=-31, \n                          max_lufs_in_db=-13, \n                          p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Normalize"},{"metadata":{"trusted":true},"cell_type":"code","source":"augment = Compose([\n    Normalize(p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### PitchShift"},{"metadata":{"trusted":true},"cell_type":"code","source":"augment = Compose([\n    PitchShift(min_semitones=-4, \n               max_semitones=4, \n               p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### PolarityInversion"},{"metadata":{"trusted":true},"cell_type":"code","source":"augment = Compose([\n    PolarityInversion(p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Resample"},{"metadata":{"trusted":true},"cell_type":"code","source":"augment = Compose([\n    Resample(min_sample_rate=8000, \n             max_sample_rate=44100, \n             p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Shift"},{"metadata":{"trusted":true},"cell_type":"code","source":"augment = Compose([\n    Shift(min_fraction=-0.5, max_fraction=0.5, rollover=True, p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### TimeMask"},{"metadata":{"trusted":true},"cell_type":"code","source":"augment = Compose([\n    TimeMask(min_band_part=0.0, \n             max_band_part=0.5, \n             fade=False, \n             p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### TimeStretch"},{"metadata":{"trusted":true},"cell_type":"code","source":"augment = Compose([\n    TimeStretch(min_rate=0.8, \n                max_rate=1.25, \n                leave_length_unchanged=True, \n                p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Trim\nThere is no trailing silence in the example. So this is not visible in this case."},{"metadata":{"trusted":true},"cell_type":"code","source":"augment = Compose([\n    Trim(top_db=20, \n         p=1.0)\n])\n\n# Augment/transform/perturb the audio data\naugmented_samples = augment(samples=wav, sample_rate=sr)\n\ncompare_signals(wav, sr, augmented_samples, sr, titles=[\"Before\", \"After\"])","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}