Modules Import
\n\n","metadata":{}},{"cell_type":"code","source":"import torchaudio\nfrom torchaudio.utils import download_asset\n\nprint(\"torchaudio's version: \", torchaudio.__version__)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:05:24.674466Z","iopub.execute_input":"2024-02-29T07:05:24.674868Z","iopub.status.idle":"2024-02-29T07:05:25.499621Z","shell.execute_reply.started":"2024-02-29T07:05:24.674837Z","shell.execute_reply":"2024-02-29T07:05:25.498062Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import albumentations \nprint(\"albumentations's version: \", albumentations.__version__)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:05:25.501791Z","iopub.execute_input":"2024-02-29T07:05:25.502683Z","iopub.status.idle":"2024-02-29T07:05:26.372457Z","shell.execute_reply.started":"2024-02-29T07:05:25.502637Z","shell.execute_reply":"2024-02-29T07:05:26.370989Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install -U albumentations","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:05:26.374235Z","iopub.execute_input":"2024-02-29T07:05:26.375030Z","iopub.status.idle":"2024-02-29T07:05:43.446876Z","shell.execute_reply.started":"2024-02-29T07:05:26.374995Z","shell.execute_reply":"2024-02-29T07:05:43.445641Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import keras_cv\nprint(\"KerasCV's version: \", keras_cv.__version__)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:05:43.449132Z","iopub.execute_input":"2024-02-29T07:05:43.449690Z","iopub.status.idle":"2024-02-29T07:05:44.001676Z","shell.execute_reply.started":"2024-02-29T07:05:43.449635Z","shell.execute_reply":"2024-02-29T07:05:44.000433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install -U torch_audiomentations","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:05:44.003194Z","iopub.execute_input":"2024-02-29T07:05:44.003605Z","iopub.status.idle":"2024-02-29T07:06:03.948654Z","shell.execute_reply.started":"2024-02-29T07:05:44.003571Z","shell.execute_reply":"2024-02-29T07:06:03.947300Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install audiomentations","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:03.951383Z","iopub.execute_input":"2024-02-29T07:06:03.951933Z","iopub.status.idle":"2024-02-29T07:06:20.093853Z","shell.execute_reply.started":"2024-02-29T07:06:03.951877Z","shell.execute_reply":"2024-02-29T07:06:20.092204Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import audiomentations\nprint(\"audiomentations's version: \", audiomentations.__version__)\n\nimport torch_audiomentations\nprint(\"torch_audiomentations's version: \", torch_audiomentations.__version__)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:20.096161Z","iopub.execute_input":"2024-02-29T07:06:20.097954Z","iopub.status.idle":"2024-02-29T07:06:20.335902Z","shell.execute_reply.started":"2024-02-29T07:06:20.097899Z","shell.execute_reply":"2024-02-29T07:06:20.334404Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import librosa","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:20.337851Z","iopub.execute_input":"2024-02-29T07:06:20.338455Z","iopub.status.idle":"2024-02-29T07:06:20.345282Z","shell.execute_reply.started":"2024-02-29T07:06:20.338411Z","shell.execute_reply":"2024-02-29T07:06:20.343785Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##librosa
","metadata":{}},{"cell_type":"markdown","source":"doc: https://github.com/librosa/librosa#Documentation\n\nLibrosa is a Python library for audio and music analysis. It provides a wide range of functions and tools for tasks such as audio loading, feature extraction, and audio manipulation.\n\nIn this notebook, I foucs on functions in `librosa.feature`, in which we can generate more features about signal. (On the other hand, this module includes many functions for processing music signal data, which will not be involved here)","metadata":{}},{"cell_type":"code","source":"# dir(librosa.feature)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - poly_features\n\n- Type of input data: **frequency domain**\n- Function's role: AddNewFeature\n\nThe function `librosa.feature.poly_features` is used to obtain the coefficients of fitting an n-th order polynomial to the columns of a spectrogram. This function computes the polynomial coefficients for each column of the spectrogram and returns a matrix where each column corresponds to the polynomial coefficients of a column of the spectrogram.\n\nWe use an n-th order polynomial to approximate each column of the spectrogram. The form of an n-th order polynomial can be represented as: y = c0 + c1x + c2x2 + … + cn*xn. Additionally, the calculation proceeds as follows:\n\nFor each column of the spectrogram:\n1. construct a vector x of length n+1 containing the index values on that column. \n2. Take the values of the spectrogram on that column as vector y. \n3. Using least squares method or other fitting algorithms, calculate the coefficients c0, c1, …, cn of the fitting polynomial by minimizing the difference between the fitting polynomial and the actual data. \n4. Store these coefficients in a matrix where each column corresponds to the polynomial coefficients of a column of the spectrogram.\n","metadata":{}},{"cell_type":"code","source":"# help(librosa.feature.poly_features)\n\np0 = librosa.feature.poly_features(S=spec_img, order=0) # (freq, time)\np1 = librosa.feature.poly_features(S=spec_img, order=1)\np2 = librosa.feature.poly_features(S=spec_img, order=2)\n\n# polynomial coefficients for each frame. \n# returns: [shape=(..., order+1, t)]","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:36.658071Z","iopub.execute_input":"2024-02-29T07:06:36.658942Z","iopub.status.idle":"2024-02-29T07:06:36.691327Z","shell.execute_reply.started":"2024-02-29T07:06:36.658902Z","shell.execute_reply":"2024-02-29T07:06:36.689523Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(nrows=3, sharex=True, figsize=(8, 8))\ntimes = librosa.times_like(p0)\nax[0].plot(times, p0[0], label='order=0', alpha=0.8)\nax[0].plot(times, p1[1], label='order=1', alpha=0.8)\nax[0].plot(times, p2[2], label='order=2', alpha=0.8)\nax[0].legend()\nax[0].set_xlabel('Time')\nax[0].label_outer()\nax[0].set(ylabel='Constant term ')\nax[1].plot(times, p1[0], label='order=1', alpha=0.8)\nax[1].plot(times, p2[1], label='order=2', alpha=0.8)\nax[1].set(ylabel='Linear term')\nax[1].label_outer()\nax[1].set_xlabel('Time')\nax[1].legend()\nax[2].plot(times, p2[0], label='order=2', alpha=0.8)\nax[2].set(ylabel='Quadratic term')\nax[2].set_xlabel('Time')\nax[2].legend()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:36.714966Z","iopub.execute_input":"2024-02-29T07:06:36.716385Z","iopub.status.idle":"2024-02-29T07:06:37.636940Z","shell.execute_reply.started":"2024-02-29T07:06:36.716311Z","shell.execute_reply":"2024-02-29T07:06:37.635562Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - rms\n\n- Type of input data: **frequency domain**\n- Function's role: AddNewFeature\n\nComputing the RMS(root_mean_square) value\n\nThe` librosa.feature.rms` function aggregates data along the frequency axis when computing the root mean square (RMS) energy for each time frame, with aggregation performed along the time axis (axis=1). In other words, it aggregates data across all frequencies to obtain an RMS energy value along the time axis.","metadata":{}},{"cell_type":"code","source":"# help(librosa.feature.rms)\n\nrms = librosa.feature.rms(S=spec_img, frame_length=spec_img.shape[-2]*2-1) # freq, time","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:37.638598Z","iopub.execute_input":"2024-02-29T07:06:37.638952Z","iopub.status.idle":"2024-02-29T07:06:37.645649Z","shell.execute_reply.started":"2024-02-29T07:06:37.638924Z","shell.execute_reply":"2024-02-29T07:06:37.644278Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(nrows=2, sharex=True)\ntimes = librosa.times_like(rms)\n\nax[0].semilogy(times, rms[0], label='RMS Energy')\nax[0].set(xticks=[])\nax[0].legend()\nax[0].label_outer()\nax[0].set_xlabel('Time')\n","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:37.647555Z","iopub.execute_input":"2024-02-29T07:06:37.647988Z","iopub.status.idle":"2024-02-29T07:06:38.370113Z","shell.execute_reply.started":"2024-02-29T07:06:37.647956Z","shell.execute_reply":"2024-02-29T07:06:38.368905Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - spectral_X\n\n- Type of input data: **frequency domain**\n- Function's role: DataAugment\n\nAll functions belows aggregate data along the time dimension and merge data along the frequency dimension.\n\nFunctions: \n- `spectral_flatness`: This function is used to compute the spectral flatness of an audio signal. Spectral flatness is a feature that describes the shape of a spectrum, measuring the uniformity of frequency components in the signal's spectrum. Spectral flatness is commonly used to analyze the harmonic structure and noise components of a signal, as well as to assess the timbral characteristics of the signal.\n- `spectral_bandwidth`: Computes the spectral bandwidth feature of an audio signal. Spectral bandwidth is typically used to describe the width of a signal's spectrum, reflecting the range of distribution of the signal's spectrum.\n- `spectral_centroid`: Computes the spectral centroid feature of an audio signal. The spectral centroid is the weighted average frequency of the spectrum, used to describe the central position of the signal's spectrum.\n- `spectral_contrast`: Computes the spectral contrast feature of an audio signal, which represents the energy differences between different frequency bands in the spectrum. Spectral contrast is a feature used to describe the dynamic range and harmonic structure of the signal's spectrum.\n- `spectral_rolloff`: Computes the spectral rolloff feature of an audio signal, which represents the frequency position where a certain percentage of energy in the spectrum is located. Spectral rolloff is commonly used to describe the high-frequency decay rate and shape of the signal's spectrum.","metadata":{}},{"cell_type":"code","source":"# help(librosa.feature.spectral_flatness) \n# help(librosa.feature.spectral_bandwidth)\n# help(librosa.feature.spectral_centroid)\n# help(librosa.feature.spectral_contrast)\n# help(librosa.feature.spectral_rolloff)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:38.372068Z","iopub.execute_input":"2024-02-29T07:06:38.372767Z","iopub.status.idle":"2024-02-29T07:06:38.377277Z","shell.execute_reply.started":"2024-02-29T07:06:38.372730Z","shell.execute_reply":"2024-02-29T07:06:38.375843Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"res = librosa.feature.spectral_flatness(S=spec_img) # assumed that the input is raw_spectrogram\n\nprint(res.shape)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:38.378610Z","iopub.execute_input":"2024-02-29T07:06:38.378960Z","iopub.status.idle":"2024-02-29T07:06:38.399867Z","shell.execute_reply.started":"2024-02-29T07:06:38.378933Z","shell.execute_reply":"2024-02-29T07:06:38.398346Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"res = librosa.feature.spectral_bandwidth(S=spec_img) # assumed that the input is raw_spectrogram\n\nprint(res.shape)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:38.402100Z","iopub.execute_input":"2024-02-29T07:06:38.402961Z","iopub.status.idle":"2024-02-29T07:06:38.415962Z","shell.execute_reply.started":"2024-02-29T07:06:38.402908Z","shell.execute_reply":"2024-02-29T07:06:38.414782Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"res = librosa.feature.spectral_centroid(S=spec_img) # assumed that the input is raw_spectrogram\n\nprint(res.shape)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:38.418426Z","iopub.execute_input":"2024-02-29T07:06:38.418924Z","iopub.status.idle":"2024-02-29T07:06:38.429715Z","shell.execute_reply.started":"2024-02-29T07:06:38.418880Z","shell.execute_reply":"2024-02-29T07:06:38.428122Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"res = librosa.feature.spectral_contrast(S=spec_img) # assumed that the input is raw_spectrogram ，need to know sample_rate\n\nprint(res.shape)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:38.443063Z","iopub.execute_input":"2024-02-29T07:06:38.443872Z","iopub.status.idle":"2024-02-29T07:06:38.455641Z","shell.execute_reply.started":"2024-02-29T07:06:38.443830Z","shell.execute_reply":"2024-02-29T07:06:38.454386Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"res = librosa.feature.spectral_rolloff(S=spec_img) # assumed that the input is raw_spectrogram , need to know sample_rate\n\nprint(res.shape)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:38.457733Z","iopub.execute_input":"2024-02-29T07:06:38.458110Z","iopub.status.idle":"2024-02-29T07:06:38.466129Z","shell.execute_reply.started":"2024-02-29T07:06:38.458080Z","shell.execute_reply":"2024-02-29T07:06:38.464632Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - mfcc","metadata":{}},{"cell_type":"markdown","source":"- Type of input data: **frequency domain**\n- Function's role: AddNewFeature\n\nMFCC (Mel Frequency Cepstral Coefficients) is a method of feature extraction based on frequencies. It combines Mel scale and cepstral analysis to extract the spectral characteristics of audio signals.\n","metadata":{}},{"cell_type":"markdown","source":"The original spectrogram and MFCC spectrogram focus on different aspects of representing audio signal features.\n\n1. The original spectrogram varies in both time and frequency, while the MFCC spectrogram mainly exhibits differences between different frequencies. \n2. **MFCC process redistributes frequencies, providing higher resolution in the lower frequency region and lower resolution in the higher frequency region.** \n\n","metadata":{}},{"cell_type":"code","source":"NAMES = ['LL','LP','RP','RR']\n\nFEATS = [['Fp1','F7','T3','T5','O1'],\n ['Fp1','F3','C3','P3','O1'],\n ['Fp2','F8','T4','T6','O2'],\n ['Fp2','F4','C4','P4','O2']]\n\nUSE_WAVELET = None\n\ndef spectrogram_from_eeg_adj(parquet_path):\n \n '''\n adjust from Chris's code: keep mel_spec as mel_spectram (don't turn to power spectram)\n '''\n \n # LOAD MIDDLE 50 SECONDS OF EEG SERIES\n eeg = pd.read_parquet(parquet_path)\n middle = (len(eeg)-10_000)//2\n eeg = eeg.iloc[middle:middle+10_000]\n \n # VARIABLE TO HOLD SPECTROGRAM\n img = np.zeros((128,256,4),dtype='float32')\n \n if display: plt.figure(figsize=(10,7))\n signals = []\n for k in range(4):\n COLS = FEATS[k]\n \n for kk in range(4):\n \n # COMPUTE PAIR DIFFERENCES\n x = eeg[COLS[kk]].values - eeg[COLS[kk+1]].values\n\n # FILL NANS\n m = np.nanmean(x)\n if np.isnan(x).mean()<1: x = np.nan_to_num(x,nan=m)\n else: x[:] = 0\n\n # DENOISE\n if USE_WAVELET:\n x = denoise(x, wavelet=USE_WAVELET)\n signals.append(x)\n\n # RAW SPECTROGRAM\n mel_spec = librosa.feature.melspectrogram(y=x, sr=200, hop_length=len(x)//256, \n n_fft=1024, n_mels=128, fmin=0, fmax=20, win_length=128)\n \n # LOG TRANSFORM\n width = (mel_spec.shape[1]//32)*32\n mel_spec = mel_spec[:,:width]\n\n # STANDARDIZE TO -1 TO 1\n mel_spec = (mel_spec+40)/40 \n img[:,:,k] += mel_spec\n \n # AVERAGE THE 4 MONTAGE DIFFERENCES\n img[:,:,k] /= 4.0\n \n return img\n","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:38.467556Z","iopub.execute_input":"2024-02-29T07:06:38.467981Z","iopub.status.idle":"2024-02-29T07:06:38.486671Z","shell.execute_reply.started":"2024-02-29T07:06:38.467951Z","shell.execute_reply":"2024-02-29T07:06:38.485424Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# adjust chris's code from \"spectrogram_from_eeg\"\neeg_img = spectrogram_from_eeg_adj('/kaggle/input/hms-harmful-brain-activity-classification/train_eegs/1000913311.parquet')\n\neeg_imgone = eeg_img[:, :, 0]\n\neeg_imgone = eeg_imgone[None, ...]","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:38.488784Z","iopub.execute_input":"2024-02-29T07:06:38.489180Z","iopub.status.idle":"2024-02-29T07:06:38.798179Z","shell.execute_reply.started":"2024-02-29T07:06:38.489148Z","shell.execute_reply":"2024-02-29T07:06:38.796531Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(10, 4))\nplt.imshow(eeg_imgone[0], origin='lower', aspect='auto')\nplt.colorbar()\nplt.title('spec raw')\nplt.xlabel('Frame')\nplt.ylabel('tk')\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:38.806026Z","iopub.execute_input":"2024-02-29T07:06:38.806834Z","iopub.status.idle":"2024-02-29T07:06:39.433179Z","shell.execute_reply.started":"2024-02-29T07:06:38.806799Z","shell.execute_reply":"2024-02-29T07:06:39.431699Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Apply mel filterbank\nn_mels_tk = 128 # Number of mel filters\nn_fft_tk = 255\nmel = librosa.filters.mel(sr=200, \n n_fft=n_fft_tk,\n n_mels=n_mels_tk, # Number of Mel filters used to project the spectrogram onto the Mel frequency axis\n fmin=0, # Minimum frequency of the Mel filter bank (in Hz)\n fmax=20 # Maximum frequency of the Mel filter bank (in Hz)\n )\n\nprint('mel.shape: ', mel.shape) # shape: (n_mels_tk, n_fft_tk//2+1) # Due to the nature of FFT, performing FFT on a signal of length N will result in N/2+1 unique frequency points\n\nmel_specgram = mel.dot(eeg_imgone[0])\n\n# Log transform\nlog_mel_specgram = librosa.amplitude_to_db(mel_specgram) # amplitude_to_db power_to_db\n\n# Compute LFCC\nn_mfcc = 12 # Number of LFCC coefficients → feature numbers\nmfcc = librosa.feature.mfcc(S=log_mel_specgram, n_mfcc=n_mfcc)\n\nprint('mfcc.shape: ', mfcc.shape)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:39.435383Z","iopub.execute_input":"2024-02-29T07:06:39.436147Z","iopub.status.idle":"2024-02-29T07:06:39.463774Z","shell.execute_reply.started":"2024-02-29T07:06:39.436102Z","shell.execute_reply":"2024-02-29T07:06:39.461906Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"When calculating MFCC, adjusting the value of the n_mfcc parameter significantly impacts the final spectrogram results.\n1. Lower n_mfcc values may be more suitable for tasks like audio classification or higher-level feature extraction in speech recognition. \n2. higher n_mfcc values may be more suitable for tasks requiring detailed spectral information, \n\nApproach to tuning n_mfcc:\n1. It is a reasonable approach to set a proper value for n_mfcc based on the ability to discover higher frequency resolution in the low-frequency region. As mentioned earlier, the low-frequency region exhibits higher frequency resolution, making lower n_mfcc values more appropriate for capturing spectral details in that region. ","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(10, 4))\nplt.imshow(mfcc, origin='lower', aspect='auto')\nplt.colorbar()\nplt.title('mfcc')\nplt.xlabel('Frame')\nplt.ylabel('fea_numbers')\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:39.466641Z","iopub.execute_input":"2024-02-29T07:06:39.467859Z","iopub.status.idle":"2024-02-29T07:06:40.101042Z","shell.execute_reply.started":"2024-02-29T07:06:39.467818Z","shell.execute_reply":"2024-02-29T07:06:40.099706Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Understanding the results obtained through the MFCC function:\n1. The features extracted by the MFCC spectrogram are primarily based on frequencies, capturing the spectral characteristics of the audio signal. \n2. **In the low-frequency region, especially near some key frequencies, higher frequency resolution may occur, meaning that the changes between adjacent frequencies are more pronounced.** \n3. For higher frequency regions, where the frequency resolution is lower and changes between adjacent frequencies are smaller, the variations between frequencies in these high-frequency areas may be less pronounced in the MFCC spectrogram.\n","metadata":{}},{"cell_type":"markdown","source":"##torchaudio
","metadata":{}},{"cell_type":"markdown","source":"TorchAudio is a Python library built on top of PyTorch, specifically designed for audio data processing and analysis.\n\ndocs: https://pytorch.org/audio/stable/index.html","metadata":{}},{"cell_type":"code","source":"# dir(torchaudio)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T10:11:34.372927Z","iopub.execute_input":"2024-02-29T10:11:34.373930Z","iopub.status.idle":"2024-02-29T10:11:34.789762Z","shell.execute_reply.started":"2024-02-29T10:11:34.373888Z","shell.execute_reply":"2024-02-29T10:11:34.788124Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - MuLawEncoding & MuLawDecoding","metadata":{}},{"cell_type":"markdown","source":"- Type of input data: **time domain**\n- Function's role: AddNewFeature\n\nOne of the main purposes of μ-law encoding is to provide higher quantization accuracy to preserve details in low-amplitude signals. In μ-law encoding, the encoded values for low-amplitude signals are larger, allowing for better differentiation between small differences in low-amplitude signals, thereby improving accuracy and fidelity in capturing low-amplitude signals.\n","metadata":{}},{"cell_type":"code","source":"# help(torchaudio.transforms.MuLawEncoding)\n\nwaveform = torch.randn(1, 160) # signal has been scaled to between -1 and 1 and\nmu_law_encoding = torchaudio.transforms.MuLawEncoding(quantization_channels=256) # a larg number means more amplitude values will be mapped to different discrete values, improving the encoding accuracy\nencoded_waveform = mu_law_encoding(waveform)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:40.102669Z","iopub.execute_input":"2024-02-29T07:06:40.103209Z","iopub.status.idle":"2024-02-29T07:06:40.169471Z","shell.execute_reply.started":"2024-02-29T07:06:40.103162Z","shell.execute_reply":"2024-02-29T07:06:40.168192Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(12, 6))\n\nplt.subplot(2, 1, 1)\nplt.plot(waveform.squeeze().numpy())\nplt.title('raw waveform')\nplt.xlabel('samples')\nplt.ylabel('amplitude')\n\nplt.subplot(2, 1, 2)\nplt.plot(encoded_waveform.squeeze().numpy())\nplt.title('waveform after encoding')\nplt.xlabel('samples')\nplt.ylabel('amplitude')\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:40.170912Z","iopub.execute_input":"2024-02-29T07:06:40.171336Z","iopub.status.idle":"2024-02-29T07:06:40.904809Z","shell.execute_reply.started":"2024-02-29T07:06:40.171300Z","shell.execute_reply":"2024-02-29T07:06:40.903629Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mu_law_decoding = torchaudio.transforms.MuLawDecoding(quantization_channels=256)\n\ndecoded_waveform = mu_law_decoding(encoded_waveform)\n\nplt.figure(figsize=(12, 3))\nplt.plot(decoded_waveform.squeeze().numpy())\nplt.title('wave after decoding')\nplt.xlabel('samples')\nplt.ylabel('amplitude')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:40.906545Z","iopub.execute_input":"2024-02-29T07:06:40.907010Z","iopub.status.idle":"2024-02-29T07:06:41.242590Z","shell.execute_reply.started":"2024-02-29T07:06:40.906969Z","shell.execute_reply":"2024-02-29T07:06:41.241284Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - SlidingWindowCmn\n\n- Type of input data: **frequency domain**\n- Function's role: AddNewFeature\n\nSliding window cepstral mean normalization is a common speech signal processing technique used to normalize cepstral coefficients of speech signals. Its purpose is to reduce volume differences between different utterances, thereby extracting more robust speech features.(Perform a sliding average calculation on the spectrogram. 😂)\n\n","metadata":{}},{"cell_type":"code","source":"# help(torchaudio.transforms.SlidingWindowCmn)\n\nSlidingWindowCmn = torchaudio.transforms.SlidingWindowCmn(cmn_window=60)\n\nres = SlidingWindowCmn(torch.tensor(spec_img)) # (..., time, freq)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:41.244114Z","iopub.execute_input":"2024-02-29T07:06:41.244735Z","iopub.status.idle":"2024-02-29T07:06:41.300273Z","shell.execute_reply.started":"2024-02-29T07:06:41.244697Z","shell.execute_reply":"2024-02-29T07:06:41.298788Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.imshow(spec_img.T, cmap='viridis', aspect='auto', origin='lower')\nplt.colorbar()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:41.302281Z","iopub.execute_input":"2024-02-29T07:06:41.302794Z","iopub.status.idle":"2024-02-29T07:06:41.789436Z","shell.execute_reply.started":"2024-02-29T07:06:41.302749Z","shell.execute_reply":"2024-02-29T07:06:41.788366Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.imshow(res.T, cmap='viridis', aspect='auto', origin='lower')\nplt.colorbar()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:41.790728Z","iopub.execute_input":"2024-02-29T07:06:41.791904Z","iopub.status.idle":"2024-02-29T07:06:42.268926Z","shell.execute_reply.started":"2024-02-29T07:06:41.791854Z","shell.execute_reply":"2024-02-29T07:06:42.267417Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - ComputeDeltas\n\n- Type of input data: **frequency domain**\n- Function's role: AddNewFeature\n\nCompute the delta coefficients of a tensor, typically a spectrogram. \n1. The function takes a tensor as input, which typically represents a sequence of audio features such as MFCCs or spectrogram frames. \n2. It then computes the delta coefficients by calculating the first-order differences between neighboring frames of the input tensor.","metadata":{}},{"cell_type":"code","source":"# help(torchaudio.transforms.ComputeDeltas)\n\nComputeDeltas = torchaudio.transforms.ComputeDeltas(win_length=5, # windows to cal delta\n mode='replicate' # \"replicate\" \"constant\"\n ) \n\nres = ComputeDeltas(torch.tensor(spec_img.T)) # input: (freq, time)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:42.270803Z","iopub.execute_input":"2024-02-29T07:06:42.271342Z","iopub.status.idle":"2024-02-29T07:06:42.354721Z","shell.execute_reply.started":"2024-02-29T07:06:42.271298Z","shell.execute_reply":"2024-02-29T07:06:42.353710Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_withT(spec_img, print_val='raw spec_img')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:42.356165Z","iopub.execute_input":"2024-02-29T07:06:42.356804Z","iopub.status.idle":"2024-02-29T07:06:42.776935Z","shell.execute_reply.started":"2024-02-29T07:06:42.356768Z","shell.execute_reply":"2024-02-29T07:06:42.773342Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_withT(res.numpy().T, print_val='spec_img after deltas')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:42.778982Z","iopub.execute_input":"2024-02-29T07:06:42.780597Z","iopub.status.idle":"2024-02-29T07:06:43.181908Z","shell.execute_reply.started":"2024-02-29T07:06:42.780535Z","shell.execute_reply":"2024-02-29T07:06:43.178352Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - Speed & SpeedPerturbation\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n \n \n- `Speed`: Adjusts waveform speed.\n - The speed of a signal usually refers to the sampling rate of the signal. The sampling rate represents the number of times a signal is sampled per second,\n- `SpeedPerturbation`: Applies the speed perturbation augmentation introduced in Audio augmentation for speech recognition\n\n","metadata":{}},{"cell_type":"code","source":"# help(torchaudio.transforms.Speed)\n\n# help(torchaudio.transforms.SpeedPerturbation)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:43.184073Z","iopub.execute_input":"2024-02-29T07:06:43.184694Z","iopub.status.idle":"2024-02-29T07:06:43.190363Z","shell.execute_reply.started":"2024-02-29T07:06:43.184648Z","shell.execute_reply":"2024-02-29T07:06:43.189298Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_rate = 44100\nduration = 5\nfrequency = 440\namplitude = 0.5\n\nt = np.linspace(0, duration, int(sample_rate * duration), endpoint=False)\nwaveform = amplitude * np.sin(2 * np.pi * frequency * t)\n\n\nwaveform_tensor = torch.tensor(waveform)\n\n\nspeed_transform = torchaudio.transforms.Speed(orig_freq=50, # sample_rate\n factor=1.1\n )\n\nres, _ = speed_transform(waveform_tensor.float())\n\nres = res.numpy()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:43.191600Z","iopub.execute_input":"2024-02-29T07:06:43.192832Z","iopub.status.idle":"2024-02-29T07:06:43.242237Z","shell.execute_reply.started":"2024-02-29T07:06:43.192781Z","shell.execute_reply":"2024-02-29T07:06:43.240757Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('len of raw waveform: ', len(waveform))\nprint()\nprint('len of waveform_after_speed', len(res))","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:43.244120Z","iopub.execute_input":"2024-02-29T07:06:43.245425Z","iopub.status.idle":"2024-02-29T07:06:43.254068Z","shell.execute_reply.started":"2024-02-29T07:06:43.245363Z","shell.execute_reply":"2024-02-29T07:06:43.252664Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - Vol\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\nAdjust volume of waveform.","metadata":{}},{"cell_type":"code","source":"# help(torchaudio.transforms.Vol)\n\nVol = torchaudio.transforms.Vol(gain=1.5, \n gain_type='amplitude' # 'amplitude', 'power', 'db'\n ) \n\nres = Vol(torch.tensor(waveform))","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:43.256111Z","iopub.execute_input":"2024-02-29T07:06:43.257114Z","iopub.status.idle":"2024-02-29T07:06:43.269733Z","shell.execute_reply.started":"2024-02-29T07:06:43.257077Z","shell.execute_reply":"2024-02-29T07:06:43.268707Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_waveform(waveform, plt_title='raw waveform')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:07:39.207978Z","iopub.execute_input":"2024-02-29T07:07:39.208572Z","iopub.status.idle":"2024-02-29T07:07:39.468786Z","shell.execute_reply.started":"2024-02-29T07:07:39.208532Z","shell.execute_reply":"2024-02-29T07:07:39.467352Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_waveform(res.numpy(), plt_title='waveform_after_gain')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:07:44.162606Z","iopub.execute_input":"2024-02-29T07:07:44.163027Z","iopub.status.idle":"2024-02-29T07:07:44.430680Z","shell.execute_reply.started":"2024-02-29T07:07:44.162996Z","shell.execute_reply":"2024-02-29T07:07:44.429094Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - TimeMasking\n\n- Type of input data: **frequency domain**\n- Function's role: DataAugment\n\nApply masking to a spectrogram in the time domain.","metadata":{}},{"cell_type":"code","source":"TimeMasking_tk = torchaudio.transforms.TimeMasking(time_mask_param=100)\nmasked = TimeMasking_tk(torch.tensor(spec_img.T)) # input (spec ,time) ","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:07:48.869300Z","iopub.execute_input":"2024-02-29T07:07:48.869720Z","iopub.status.idle":"2024-02-29T07:07:48.888705Z","shell.execute_reply.started":"2024-02-29T07:07:48.869689Z","shell.execute_reply":"2024-02-29T07:07:48.887336Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_withlogwithT(masked.T.numpy(), print_val='spec_img after TimeMasking')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:07:50.406055Z","iopub.execute_input":"2024-02-29T07:07:50.406544Z","iopub.status.idle":"2024-02-29T07:07:50.803372Z","shell.execute_reply.started":"2024-02-29T07:07:50.406511Z","shell.execute_reply":"2024-02-29T07:07:50.802349Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - TimeStretch\n\n- Type of input data: **frequency domain**\n- Function's role: DataAugment\n\nStretch stft in time without modifying pitch for a given rate.","metadata":{}},{"cell_type":"code","source":"TimeStretch_tk = torchaudio.transforms.TimeStretch(n_freq=100)\nprint(spec_img.T.shape)\n\nTimeStretch_res = TimeStretch_tk(torch.tensor(spec_img.T), 0.9)\nprint(TimeStretch_res.real.numpy().shape)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:07:55.337445Z","iopub.execute_input":"2024-02-29T07:07:55.337872Z","iopub.status.idle":"2024-02-29T07:07:55.368596Z","shell.execute_reply.started":"2024-02-29T07:07:55.337841Z","shell.execute_reply":"2024-02-29T07:07:55.367225Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_withlogwithT(TimeStretch_res.T.real.numpy(), print_val='spec_img after TimeStretch')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:07:58.235976Z","iopub.execute_input":"2024-02-29T07:07:58.237538Z","iopub.status.idle":"2024-02-29T07:07:58.683018Z","shell.execute_reply.started":"2024-02-29T07:07:58.237483Z","shell.execute_reply":"2024-02-29T07:07:58.681348Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - SpectralCentroid\n\n- Type of input data: **time domain**\n- Function's role: AddNewFeature\n\nCompute the spectral centroid for each channel along the time axis.","metadata":{}},{"cell_type":"code","source":"SpectralCentroid = torchaudio.transforms.SpectralCentroid(sample_rate=200)\n\nres = SpectralCentroid(torch.tensor(waveform))","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:08:01.448889Z","iopub.execute_input":"2024-02-29T07:08:01.449375Z","iopub.status.idle":"2024-02-29T07:08:01.523090Z","shell.execute_reply.started":"2024-02-29T07:08:01.449339Z","shell.execute_reply":"2024-02-29T07:08:01.521543Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('raw waveform.shape: ', waveform.shape)\n\nprint('waveform_after_SpectralCentroid.shape: ', res.shape) ","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:08:03.687487Z","iopub.execute_input":"2024-02-29T07:08:03.687962Z","iopub.status.idle":"2024-02-29T07:08:03.695280Z","shell.execute_reply.started":"2024-02-29T07:08:03.687931Z","shell.execute_reply":"2024-02-29T07:08:03.693815Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - FrequencyMasking\n\nType of input data: **frequency domain**\n\nApply masking to a spectrogram in the frequency domain.","metadata":{}},{"cell_type":"code","source":"masking = torchaudio.transforms.FrequencyMasking(freq_mask_param=80)\n\nmasked = masking(torch.tensor(spec_img).T) # (spec, time)\n\nplot_withlogwithT(masked.T.real.numpy(), print_val='spec_img after TimeStretch')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:08:05.692583Z","iopub.execute_input":"2024-02-29T07:08:05.693026Z","iopub.status.idle":"2024-02-29T07:08:06.141274Z","shell.execute_reply.started":"2024-02-29T07:08:05.692993Z","shell.execute_reply":"2024-02-29T07:08:06.139806Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - Resample\n\n- Type of input data: **frequency domain**\n- Function's role: DataAugment\n\nResample a signal from one frequency to another. A resampling method can be given.","metadata":{}},{"cell_type":"code","source":"# help(torchaudio.transforms.Resample)\n\n# Resample a signal from one frequency to another. A resampling method can be given.\n\nResample = torchaudio.transforms.Resample(orig_freq=44100, new_freq=10000)\n\nres = Resample(torch.tensor(waveform).float())","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:08:08.469010Z","iopub.execute_input":"2024-02-29T07:08:08.469550Z","iopub.status.idle":"2024-02-29T07:08:08.484195Z","shell.execute_reply.started":"2024-02-29T07:08:08.469514Z","shell.execute_reply":"2024-02-29T07:08:08.483164Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('len of wave_after_resample: ', len(res))","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:08:10.877719Z","iopub.execute_input":"2024-02-29T07:08:10.878133Z","iopub.status.idle":"2024-02-29T07:08:10.885781Z","shell.execute_reply.started":"2024-02-29T07:08:10.878102Z","shell.execute_reply":"2024-02-29T07:08:10.884308Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - SpecAugment\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\nApply time and frequency masking to a spectrogram.","metadata":{}},{"cell_type":"code","source":"# help(torchaudio.transforms.SpecAugment)\n\nSpecAugment = torchaudio.transforms.SpecAugment(n_time_masks=3, time_mask_param=15, n_freq_masks=3, freq_mask_param=20, p=1, zero_masking=False)\n\nres = SpecAugment(torch.tensor(spec_img))","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:08:12.161160Z","iopub.execute_input":"2024-02-29T07:08:12.162506Z","iopub.status.idle":"2024-02-29T07:08:12.175401Z","shell.execute_reply.started":"2024-02-29T07:08:12.162459Z","shell.execute_reply":"2024-02-29T07:08:12.173917Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_withlogwithT(res, print_val='spec_img after SpecAugment')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:08:14.346878Z","iopub.execute_input":"2024-02-29T07:08:14.347356Z","iopub.status.idle":"2024-02-29T07:08:15.564496Z","shell.execute_reply.started":"2024-02-29T07:08:14.347323Z","shell.execute_reply":"2024-02-29T07:08:15.563160Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##keras_cv
","metadata":{}},{"cell_type":"markdown","source":"https://keras.io/api/keras_cv/layers/\n\n`keras_cv` is not specifically designed for signal data processing but rather for image data processing. However, we can utilize its methods to perform data augmentation on spectrograms.","metadata":{}},{"cell_type":"markdown","source":"## - FourierMix\n\n- Type of input data: **frequency domain(image)**\n- Function's role: DataAugment\n\nFourierMix implements the FMix data augmentation technique. FMix generates random masks by applying Fourier transform to the image, and then mixes the original image with these masks to generate a new enhanced image.\n","metadata":{}},{"cell_type":"code","source":"(images, labels), _ = keras.datasets.cifar10.load_data()\nlabels = tf.one_hot(labels, 10)\nlabels = tf.cast(tf.squeeze(labels), tf.float32)\nsample_size = 4","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:08:18.596072Z","iopub.execute_input":"2024-02-29T07:08:18.596742Z","iopub.status.idle":"2024-02-29T07:08:25.497021Z","shell.execute_reply.started":"2024-02-29T07:08:18.596700Z","shell.execute_reply":"2024-02-29T07:08:25.495882Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_images = images[:sample_size]\nsample_labels = labels[:sample_size]","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:09:29.010985Z","iopub.execute_input":"2024-02-29T07:09:29.011441Z","iopub.status.idle":"2024-02-29T07:09:29.018383Z","shell.execute_reply.started":"2024-02-29T07:09:29.011409Z","shell.execute_reply":"2024-02-29T07:09:29.016728Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# help(keras_cv.layers.FourierMix)\n\nfouriermix = keras_cv.layers.FourierMix()\noutput = fouriermix({'images': tf.convert_to_tensor(sample_images), 'labels': sample_labels})","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:12:51.913047Z","iopub.execute_input":"2024-02-29T07:12:51.913480Z","iopub.status.idle":"2024-02-29T07:12:52.864648Z","shell.execute_reply.started":"2024-02-29T07:12:51.913449Z","shell.execute_reply":"2024-02-29T07:12:52.863528Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axs = plt.subplots(2, 2, figsize=(8, 8))\n\nfor i in range(4):\n row = i // 2\n col = i % 2\n axs[row, col].imshow(images[i, :, :, -1])\n axs[row, col].set_title(f'Image {i+1}')\n axs[row, col].axis('off')\n\nplt.suptitle('Four origin Images Display channel(3)')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:12:56.219298Z","iopub.execute_input":"2024-02-29T07:12:56.219705Z","iopub.status.idle":"2024-02-29T07:12:56.707031Z","shell.execute_reply.started":"2024-02-29T07:12:56.219676Z","shell.execute_reply":"2024-02-29T07:12:56.705480Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axs = plt.subplots(2, 2, figsize=(8, 8))\n\nfor i in range(4):\n row = i // 2\n col = i % 2\n axs[row, col].imshow(output['images'][i, :, :, -1])\n axs[row, col].set_title(f'Image {i+1}')\n axs[row, col].axis('off')\n\nplt.suptitle('Four output Images Display channel(3)')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:13:03.270031Z","iopub.execute_input":"2024-02-29T07:13:03.271318Z","iopub.status.idle":"2024-02-29T07:13:03.749614Z","shell.execute_reply.started":"2024-02-29T07:13:03.271242Z","shell.execute_reply":"2024-02-29T07:13:03.748320Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - Mosaic\n\n- Type of input data: **frequency domain(image)**\n- Function's role: DataAugment\n\nMosaic data augmentation first takes 4 images from the batch and makes agrid. After that based on the offset, a crop is taken to form the mosaic image. Labels are in the same ratio as the area of their images in the output image. Bounding boxes are translated according to the position of the 4 images. ","metadata":{}},{"cell_type":"code","source":"(images, labels), _ = keras.datasets.cifar10.load_data() \nlabels = tf.one_hot(labels, 10) \nlabels = tf.cast(tf.squeeze(labels), tf.float32) \nsample_size = 4 \nsample_images = images[:sample_size] \nsample_labels = labels[:sample_size]\n\n\nmosaic = keras_cv.layers.Mosaic()\noutput = mosaic({'images': tf.convert_to_tensor(sample_images), 'labels': sample_labels})\n\nprint('sample_images.shape: ',sample_images.shape)\nprint('')\nprint('output_images.shape: ',output['images'].shape)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:13:39.574859Z","iopub.execute_input":"2024-02-29T07:13:39.575288Z","iopub.status.idle":"2024-02-29T07:13:40.575843Z","shell.execute_reply.started":"2024-02-29T07:13:39.575242Z","shell.execute_reply":"2024-02-29T07:13:40.574449Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axs = plt.subplots(2, 2, figsize=(8, 8))\n\nfor i in range(4):\n row = i // 2\n col = i % 2\n axs[row, col].imshow(images[i, :, :, -1])\n axs[row, col].set_title(f'Image {i+1}')\n axs[row, col].axis('off')\n\nplt.suptitle('Four origin Images Display channel(3)')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:13:44.931868Z","iopub.execute_input":"2024-02-29T07:13:44.932298Z","iopub.status.idle":"2024-02-29T07:13:45.404913Z","shell.execute_reply.started":"2024-02-29T07:13:44.932242Z","shell.execute_reply":"2024-02-29T07:13:45.403698Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axs = plt.subplots(2, 2, figsize=(8, 8))\n\nfor i in range(4):\n row = i // 2\n col = i % 2\n axs[row, col].imshow(output['images'][i, :, :, -1])\n axs[row, col].set_title(f'Image {i+1}')\n axs[row, col].axis('off')\n\nplt.suptitle('Four output Images Display channel(3)')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:13:47.664631Z","iopub.execute_input":"2024-02-29T07:13:47.665080Z","iopub.status.idle":"2024-02-29T07:13:48.149010Z","shell.execute_reply.started":"2024-02-29T07:13:47.665049Z","shell.execute_reply":"2024-02-29T07:13:48.147769Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - Mixup\n\n- Type of input data: **frequency domain(image)**\n- Function's role: DataAugment\n\nMixUp implements the MixUp data augmentation technique.","metadata":{}},{"cell_type":"code","source":"(images, labels), _ = keras.datasets.cifar10.load_data()\n\nsample_size = 2 # sample_images = images[:sample_size] sample_labels = labels[:sample_size]\nsample_images = images[:sample_size] \nsample_labels = labels[:sample_size]\n\nsample_labels_oht = tf.cast(tf.one_hot(sample_labels, 10), tf.float32) # to use mixup, Labels must be floating-point and one-hot encoded","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:14:01.944829Z","iopub.execute_input":"2024-02-29T07:14:01.945303Z","iopub.status.idle":"2024-02-29T07:14:02.957498Z","shell.execute_reply.started":"2024-02-29T07:14:01.945244Z","shell.execute_reply":"2024-02-29T07:14:02.956442Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# help(keras_cv.layers.MixUp)\n\nmixup = keras_cv.layers.MixUp(alpha=0.5, # Float between 0 and 1. This controls the shape of the distribution from which the smoothing values are sampled. Defaults to 0.2,\n seed=33)\naugmented_data = mixup(\n {'images': tf.convert_to_tensor(sample_images), 'labels': sample_labels_oht}\n)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:14:33.399457Z","iopub.execute_input":"2024-02-29T07:14:33.399904Z","iopub.status.idle":"2024-02-29T07:14:33.421981Z","shell.execute_reply.started":"2024-02-29T07:14:33.399870Z","shell.execute_reply":"2024-02-29T07:14:33.420736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axs = plt.subplots(1, 2, figsize=(8, 4)) \n\nfor i in range(2): \n axs[i].imshow(sample_images[i, :, :, -1])\n axs[i].set_title(f'Image {i+1}')\n axs[i].axis('off')\n\nplt.suptitle('Two raw Images Display channel(3)') \nplt.show()\n\n# #######################################################\n\nfig, axs = plt.subplots(1, 2, figsize=(8, 4)) \n\nfor i in range(2): \n axs[i].imshow(augmented_data['images'][i, :, :, -1])\n axs[i].set_title(f'Image {i+1}')\n axs[i].axis('off')\n\nplt.suptitle('Two output Images Display channel(3)')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:14:37.071640Z","iopub.execute_input":"2024-02-29T07:14:37.072090Z","iopub.status.idle":"2024-02-29T07:14:37.581827Z","shell.execute_reply.started":"2024-02-29T07:14:37.072057Z","shell.execute_reply":"2024-02-29T07:14:37.580401Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - CutMix\n\n- Type of input data: **frequency domain(image)**\n- Function's role: DataAugment\n\nCutmix generates new training samples by randomly cropping and mixing two different images in the input image. This helps the model to better generalize and improve its performance.","metadata":{}},{"cell_type":"code","source":"(images, labels), _ = keras.datasets.cifar10.load_data()\n\nsample_size = 2 #\nsample_images = images[:sample_size] \nsample_labels = labels[:sample_size]\n\nsample_labels_oht = tf.cast(tf.one_hot(sample_labels, 10), tf.float32) # Labels must be floating-point and one-hot encoded\n\n# mixup = keras_cv.layers.MixUp(alpha=0.5, seed=42)\ncutmix = keras_cv.layers.CutMix(alpha=0.5, seed=42)\naugmented_data = cutmix(\n {'images': tf.convert_to_tensor(sample_images), 'labels': sample_labels_oht}\n)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:14:57.152239Z","iopub.execute_input":"2024-02-29T07:14:57.152693Z","iopub.status.idle":"2024-02-29T07:14:58.174409Z","shell.execute_reply.started":"2024-02-29T07:14:57.152662Z","shell.execute_reply":"2024-02-29T07:14:58.172764Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axs = plt.subplots(1, 2, figsize=(8, 4)) \n\nfor i in range(2): \n axs[i].imshow(sample_images[i, :, :, -1])\n axs[i].set_title(f'Image {i+1}')\n axs[i].axis('off')\n\nplt.suptitle('Two raw Images Display channel(3)') \nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:14:58.869080Z","iopub.execute_input":"2024-02-29T07:14:58.870101Z","iopub.status.idle":"2024-02-29T07:14:59.082983Z","shell.execute_reply.started":"2024-02-29T07:14:58.870063Z","shell.execute_reply":"2024-02-29T07:14:59.081484Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axs = plt.subplots(1, 2, figsize=(8, 4)) #\n\nfor i in range(2): \n axs[i].imshow(augmented_data['images'][i, :, :, -1])\n axs[i].set_title(f'Image {i+1}')\n axs[i].axis('off')\n\nplt.suptitle('Two output Images Display channel(3)') # \nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:01.822239Z","iopub.execute_input":"2024-02-29T07:15:01.822759Z","iopub.status.idle":"2024-02-29T07:15:02.050789Z","shell.execute_reply.started":"2024-02-29T07:15:01.822720Z","shell.execute_reply":"2024-02-29T07:15:02.049804Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - RandomCrop\n\n- Type of input data: **frequency domain(image)**\n- Function's role: DataAugment\n\nA preprocessing layer which randomly crops images.\n\nDuring training, \n1. this layer will randomly choose a location to crop images down to a target size. \n2. The layer will crop all the images in the same batch to the same cropping location.\n\nAt inference time, and during training \n1. if an input image is smaller than the target size, the input will be resized and cropped so as to return the largest possible window in the image that matches the target aspect ratio. \n2. If you need to apply random cropping at inference time, set training to True when calling the layer.","metadata":{}},{"cell_type":"code","source":"# help(keras_cv.layers.RandomCrop) # RandomCrop(height, width, seed=None, bounding_box_format=None, **kwargs)\n\nRandomCrop = keras_cv.layers.RandomCrop(height=200, width=50, seed=42)\n\nres = RandomCrop(spec_img_3D)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:13.932838Z","iopub.execute_input":"2024-02-29T07:15:13.933350Z","iopub.status.idle":"2024-02-29T07:15:13.991076Z","shell.execute_reply.started":"2024-02-29T07:15:13.933317Z","shell.execute_reply":"2024-02-29T07:15:13.989555Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_withlogwithT(res[:, :, -1], print_val='RandomCrop')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:16.391840Z","iopub.execute_input":"2024-02-29T07:15:16.392276Z","iopub.status.idle":"2024-02-29T07:15:16.790207Z","shell.execute_reply.started":"2024-02-29T07:15:16.392223Z","shell.execute_reply":"2024-02-29T07:15:16.788715Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - RandomFlip\n\n- Type of input data: **frequency domain(image)**\n- Function's role: DataAugment\n\nA preprocessing layer which randomly flips images ","metadata":{}},{"cell_type":"code","source":"# help(keras_cv.layers.RandomFlip)\n\nrandom_flip_layer = keras_cv.layers.RandomFlip(mode='horizontal', # 'vertical' ; 'horizontal'\n rate=1, \n seed=42) \n\noutput_image = random_flip_layer(spec_img_3D)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:19.014541Z","iopub.execute_input":"2024-02-29T07:15:19.015022Z","iopub.status.idle":"2024-02-29T07:15:19.054689Z","shell.execute_reply.started":"2024-02-29T07:15:19.014989Z","shell.execute_reply":"2024-02-29T07:15:19.052923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_withlogwithT(spec_img_3D[:, :, -1], print_val='raw spec_img')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:22.468426Z","iopub.execute_input":"2024-02-29T07:15:22.468923Z","iopub.status.idle":"2024-02-29T07:15:22.867390Z","shell.execute_reply.started":"2024-02-29T07:15:22.468888Z","shell.execute_reply":"2024-02-29T07:15:22.866044Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_withlogwithT(output_image[:, :, -1], print_val='spec_img after RandomFlip')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:25.113445Z","iopub.execute_input":"2024-02-29T07:15:25.113861Z","iopub.status.idle":"2024-02-29T07:15:25.527496Z","shell.execute_reply.started":"2024-02-29T07:15:25.113830Z","shell.execute_reply":"2024-02-29T07:15:25.526128Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - RandomCutout\n\n- Type of input data: **frequency domain(image)**\n- Function's role: DataAugment\n\nRandomly cut out rectangles from images and fill them. Image augmentation layers are expecting inputs to be rank 3 (HWC) or 4D (NHWC) tensors","metadata":{}},{"cell_type":"code","source":"# help(keras_cv.layers.RandomCutout) \n\nrandom_cutout_layer = keras_cv.layers.RandomCutout(\n height_factor=(1.0, 1.0), # percentage tuples\n width_factor=(0.06, 0.1), \n fill_mode='constant', # fill_mode: constant, gaussian_noise \n seed=42)\n\noutput_image = random_cutout_layer(spec_img_3D)\n\nplot_withlogwithT(output_image[:, :, -1], print_val='spec_img after RandomCutout')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:28.229808Z","iopub.execute_input":"2024-02-29T07:15:28.231658Z","iopub.status.idle":"2024-02-29T07:15:28.731688Z","shell.execute_reply.started":"2024-02-29T07:15:28.231589Z","shell.execute_reply":"2024-02-29T07:15:28.730670Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - RandomRotation\n\n- Type of input data: **frequency domain(image)**\n- Function's role: DataAugment\n\nA preprocessing layer which randomly rotates images. ","metadata":{}},{"cell_type":"code","source":"# help(keras_cv.layers.RandomRotation) \n\nrandom_rotation_layer = keras_cv.layers.RandomRotation(factor=0.05, \n fill_mode='reflect' # one of {\"constant\", \"reflect\", \"wrap\", \"nearest\"}\n ) \n\noutput_image = random_rotation_layer(spec_img_3D)\n\nplot_withlogwithT(output_image[:, :, -1], print_val='spec_img after RandomRotation')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:30.957911Z","iopub.execute_input":"2024-02-29T07:15:30.959515Z","iopub.status.idle":"2024-02-29T07:15:31.442948Z","shell.execute_reply.started":"2024-02-29T07:15:30.959451Z","shell.execute_reply":"2024-02-29T07:15:31.441662Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - RandomCutout\n\n- Type of input data: **frequency domain(image)**\n- Function's role: DataAugment\n\nRandomly cut out rectangles from images and fill them.\n","metadata":{}},{"cell_type":"code","source":"# help(keras_cv.layers.RandomCutout)\n\nrandom_cutout = keras_cv.layers.RandomCutout(height_factor=0.1, \n width_factor=0.5, \n fill_mode='constant', # one of {\"constant\", \"gaussian_noise\"}\n seed=42\n )\noutput_image = random_cutout(spec_img_3D)\n\nplot_withlogwithT(output_image[:, :, -1], print_val='spec_img after RandomCutout')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:32.848573Z","iopub.execute_input":"2024-02-29T07:15:32.849042Z","iopub.status.idle":"2024-02-29T07:15:33.344108Z","shell.execute_reply.started":"2024-02-29T07:15:32.849007Z","shell.execute_reply":"2024-02-29T07:15:33.342556Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - RandomZoom\n\n- Type of input data: **frequency domain(image)**\n- Function's role: DataAugment\n\nA preprocessing layer which randomly zooms images. This layer will randomly zoom in or out on each axis of an image independently, filling empty space according to `fill_mode`.","metadata":{}},{"cell_type":"code","source":"# help(keras_cv.layers.RandomZoom)\n\nRandomZoom = keras_cv.layers.RandomZoom(height_factor=(-0.2, 0.3), \n width_factor=(-0.2, 0.3), \n fill_mode=\"nearest\" # {\"constant\", \"reflect\", \"wrap\", \"nearest\"}\n )\n\nres = RandomZoom(spec_img_3D)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:35.453746Z","iopub.execute_input":"2024-02-29T07:15:35.454374Z","iopub.status.idle":"2024-02-29T07:15:35.484475Z","shell.execute_reply.started":"2024-02-29T07:15:35.454329Z","shell.execute_reply":"2024-02-29T07:15:35.482810Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_withT(res.numpy()[:, :, -1])","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:36.874441Z","iopub.execute_input":"2024-02-29T07:15:36.875659Z","iopub.status.idle":"2024-02-29T07:15:37.266056Z","shell.execute_reply.started":"2024-02-29T07:15:36.875619Z","shell.execute_reply":"2024-02-29T07:15:37.264641Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - RandomTranslation\n\n- Type of input data: **frequency domain(image)**\n- Function's role: DataAugment\n\nA preprocessing layer which randomly translates images.","metadata":{}},{"cell_type":"code","source":"# help(keras_cv.layers.RandomTranslation) #\n\n# height_factor=(-0.2, 0.3)` results in an output shifted by a random amount in the range `[-20%, +30%]\nRandomTranslation = keras_cv.layers.RandomTranslation(height_factor=(-0.2, 0.3), \n width_factor=(-0.2, 0.3), \n fill_mode=\"nearest\" # {\"constant\", \"reflect\", \"wrap\", \"nearest\"}\n )\n\nres = RandomTranslation(spec_img_3D)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:38.689747Z","iopub.execute_input":"2024-02-29T07:15:38.690190Z","iopub.status.idle":"2024-02-29T07:15:38.713503Z","shell.execute_reply.started":"2024-02-29T07:15:38.690157Z","shell.execute_reply":"2024-02-29T07:15:38.711417Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_withT(res.numpy()[:, :, -1])","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:40.000369Z","iopub.execute_input":"2024-02-29T07:15:40.001480Z","iopub.status.idle":"2024-02-29T07:15:40.395289Z","shell.execute_reply.started":"2024-02-29T07:15:40.001437Z","shell.execute_reply":"2024-02-29T07:15:40.394011Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - RandomAspectRatio\n\n- Type of input data: **frequency domain(image)**\n- Function's role: DataAugment\n\nAn image enhancement technique used to randomly distort the aspect ratio of an image. Changing the aspect ratio of an image","metadata":{}},{"cell_type":"code","source":"# help(keras_cv.layers.RandomAspectRatio)\n\nrandom_aspect_ratio = keras_cv.layers.RandomAspectRatio(factor=0.2)\n\nres = random_aspect_ratio(spec_img_3D)\n\nprint('shape of specimg_after_RandomAspectRatio: ',res[:,:, -1].shape)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:42.442541Z","iopub.execute_input":"2024-02-29T07:15:42.443008Z","iopub.status.idle":"2024-02-29T07:15:42.503900Z","shell.execute_reply.started":"2024-02-29T07:15:42.442975Z","shell.execute_reply":"2024-02-29T07:15:42.502297Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - RandomCropAndResize\n\n- Type of input data: **frequency domain(image)**\n- Function's role: DataAugment\n\nRandomly crops a part of an image and resizes it to provided size. This implementation takes an intuitive approach, where we crop the images to a random height and width, and then resize them.","metadata":{}},{"cell_type":"code","source":"# help(keras_cv.layers.RandomCropAndResize)\n\nrandom_flip_layer = keras_cv.layers.RandomCropAndResize(\n target_size=(300,50), # A tuple of two integers used as the target size to ultimately crop images to\n crop_area_factor=(0.8,1.0), # The ratio of area of the cropped part to that of original image is sampled using this factor\n aspect_ratio_factor=(3/4.0,4/3.0) # the aspect ratio sampled represents a value to distort the aspect ratio by. (3/4, 4/3) (1.0, 1.0)\n\n) \n\noutput_image = random_flip_layer(spec_img_3D)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:44.054449Z","iopub.execute_input":"2024-02-29T07:15:44.055857Z","iopub.status.idle":"2024-02-29T07:15:44.083787Z","shell.execute_reply.started":"2024-02-29T07:15:44.055816Z","shell.execute_reply":"2024-02-29T07:15:44.082334Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_withlogwithT(spec_img_3D[:, :, -1], print_val='raw spec_img')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:46.226880Z","iopub.execute_input":"2024-02-29T07:15:46.227329Z","iopub.status.idle":"2024-02-29T07:15:46.697095Z","shell.execute_reply.started":"2024-02-29T07:15:46.227297Z","shell.execute_reply":"2024-02-29T07:15:46.695724Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_withlogwithT(output_image[:, :, -1], print_val='random_flip_layer')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:49.240441Z","iopub.execute_input":"2024-02-29T07:15:49.240909Z","iopub.status.idle":"2024-02-29T07:15:49.687129Z","shell.execute_reply.started":"2024-02-29T07:15:49.240878Z","shell.execute_reply":"2024-02-29T07:15:49.685593Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - RandomShear\n\n- Type of input data: **frequency domain(image)**\n- Function's role: DataAugment\n\nA preprocessing layer which randomly shears images.\n\nThis layer will apply random shearings to each image, filling empty space according to fill_mode.\n\nInput pixel values can be of any range and any data type.","metadata":{}},{"cell_type":"code","source":"# help(keras_cv.layers.RandomShear)\n\nrandom_shear_layer = keras_cv.layers.RandomShear(0.5, 0.5) \noutput_image = random_shear_layer(spec_img_3D)\n\nplot_withlogwithT(output_image[:, :, -1], print_val='RandomShear')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:52.101613Z","iopub.execute_input":"2024-02-29T07:15:52.102072Z","iopub.status.idle":"2024-02-29T07:15:52.583034Z","shell.execute_reply.started":"2024-02-29T07:15:52.102033Z","shell.execute_reply":"2024-02-29T07:15:52.581665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##audiomentations
","metadata":{}},{"cell_type":"markdown","source":"`audiomentations` is a Python library designed for audio data augmentation. It offers a comprehensive set of audio transformation techniques that can be applied to audio signals\n\n`torch_audiomentations` is inspired by `audiomentations` and contains similiar functions\n\ndocs:\n- https://github.com/iver56/audiomentations\n- https://iver56.github.io/audiomentations/\n- https://pytorch.org/audio/stable/index.html\n\n","metadata":{}},{"cell_type":"markdown","source":"## - PitchShift\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\nPitch shift the sound up or down without changing the tempo\n\nA converter for audio pitch shifting. It can alter the pitch of an audio by raising or lowering it by a specified number of semitones. The input object for the function is the time-domain waveform data. It directly performs pitch shifting on the audio waveform without converting it into spectral domain data.\n1. The PitchShift function operates on time-domain data primarily by adjusting the pitch to change the frequency of the audio.\n2. The parameter min_semitones controls the minimum unit of pitch variation, but in the implementation process, it may involve amplitude adjustment.\n3. When the pitch changes, the frequency also changes, which may result in amplitude variations in the audio.\n4. Therefore, when achieving PitchShift by adjusting the pitch, it may involve adjusting the amplitude to maintain the overall quality and balance of the audio. Thus, it can be understood that while adjusting the pitch, there may also be some degree of amplitude adjustment.\"\n \n","metadata":{}},{"cell_type":"code","source":"PitchShift = audiomentations.PitchShift(min_semitones=-10, # Minimum semitones to shift. Negative number means shift down. \n max_semitones=0, # Maximum semitones to shift. Positive number means shift up. \n p=1) \n\nres = PitchShift(waveform, sample_rate=44100)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:54.827927Z","iopub.execute_input":"2024-02-29T07:15:54.828370Z","iopub.status.idle":"2024-02-29T07:15:56.782135Z","shell.execute_reply.started":"2024-02-29T07:15:54.828340Z","shell.execute_reply":"2024-02-29T07:15:56.780861Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_waveform(waveform, sample_split=250, plt_title='raw waveform')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:56.784342Z","iopub.execute_input":"2024-02-29T07:15:56.784761Z","iopub.status.idle":"2024-02-29T07:15:57.074305Z","shell.execute_reply.started":"2024-02-29T07:15:56.784728Z","shell.execute_reply":"2024-02-29T07:15:57.073037Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_waveform(res, sample_split=250, plt_title='waveform after PitchShift')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:57.321102Z","iopub.execute_input":"2024-02-29T07:15:57.321548Z","iopub.status.idle":"2024-02-29T07:15:57.606456Z","shell.execute_reply.started":"2024-02-29T07:15:57.321517Z","shell.execute_reply":"2024-02-29T07:15:57.605315Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - Shift\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\nShift the samples forwards or backwards, with or without rollover\n","metadata":{"execution":{"iopub.status.busy":"2024-02-25T09:45:42.812797Z","iopub.execute_input":"2024-02-25T09:45:42.813373Z","iopub.status.idle":"2024-02-25T09:45:42.820683Z","shell.execute_reply.started":"2024-02-25T09:45:42.813342Z","shell.execute_reply":"2024-02-25T09:45:42.819994Z"}}},{"cell_type":"code","source":"Shift = audiomentations.Shift(min_shift=0, # Minimum amount of shifting in time.\n max_shift=0.01, # Maximum amount of shifting in time.\n p=1, \n shift_unit='fraction', # 'fraction'(Fraction of the total sound length), 'samples'(Number of audio samples), 'seconds'(Number of seconds)\n rollover=False, # rollover=False results in an empty space (with zeroes).\n fade_duration=0.0)\n\nres = Shift(waveform, sample_rate=44100)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:15:59.984502Z","iopub.execute_input":"2024-02-29T07:15:59.985334Z","iopub.status.idle":"2024-02-29T07:15:59.992036Z","shell.execute_reply.started":"2024-02-29T07:15:59.985293Z","shell.execute_reply":"2024-02-29T07:15:59.990553Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_waveform(waveform, sample_split=1000, plt_title='raw waveform')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:01.083140Z","iopub.execute_input":"2024-02-29T07:16:01.083984Z","iopub.status.idle":"2024-02-29T07:16:01.392794Z","shell.execute_reply.started":"2024-02-29T07:16:01.083923Z","shell.execute_reply":"2024-02-29T07:16:01.391361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_waveform(res, sample_split=1000, plt_title='waveform after Shift')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:03.840116Z","iopub.execute_input":"2024-02-29T07:16:03.840859Z","iopub.status.idle":"2024-02-29T07:16:04.127039Z","shell.execute_reply.started":"2024-02-29T07:16:03.840822Z","shell.execute_reply":"2024-02-29T07:16:04.125540Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - TimeStretch\n\nType of input data: **time domain**\n\nTime stretch the signal without changing the pitch","metadata":{"execution":{"iopub.status.busy":"2024-02-25T09:45:44.784462Z","iopub.execute_input":"2024-02-25T09:45:44.784742Z","iopub.status.idle":"2024-02-25T09:45:44.798181Z","shell.execute_reply.started":"2024-02-25T09:45:44.784720Z","shell.execute_reply":"2024-02-25T09:45:44.797142Z"}}},{"cell_type":"markdown","source":"If you set leave_length_unchanged to True, the function will attempt to maintain the length of the waveform unchanged during time stretching. This means that even if time stretching is applied, the waveform’s number of samples will still remain the same as the original waveform. This may result in some changes, such as a potential variation in the audio signal’s sampling rate to maintain the length of the waveform.","metadata":{}},{"cell_type":"code","source":"# help(audiomentations.TimeStretch)\n\nTimeStretch = audiomentations.TimeStretch(min_rate=1, max_rate=3, p=1, leave_length_unchanged=True)\n\nres = TimeStretch(waveform, sample_rate=44100)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:06.552665Z","iopub.execute_input":"2024-02-29T07:16:06.553115Z","iopub.status.idle":"2024-02-29T07:16:06.659001Z","shell.execute_reply.started":"2024-02-29T07:16:06.553084Z","shell.execute_reply":"2024-02-29T07:16:06.657849Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('waveform.mean(): {:.6f}'.format(waveform.mean()))\nprint('waveform.std(): {:.6f}'.format(waveform.std()))\n\nprint('')\nprint('res.mean(): {:.6f}'.format(res.mean()))\nprint('res.std(): {:.6f}'.format(res.std()))","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:08.698649Z","iopub.execute_input":"2024-02-29T07:16:08.699072Z","iopub.status.idle":"2024-02-29T07:16:08.710052Z","shell.execute_reply.started":"2024-02-29T07:16:08.699044Z","shell.execute_reply":"2024-02-29T07:16:08.708626Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - Clip\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\nClip audio by specified values. e.g. set a_min=-1.0 and a_max=1.0 to ensure that no samples in the audio exceed that extent.","metadata":{"execution":{"iopub.status.busy":"2024-02-25T09:45:44.877382Z","iopub.execute_input":"2024-02-25T09:45:44.877618Z","iopub.status.idle":"2024-02-25T09:45:44.886177Z","shell.execute_reply.started":"2024-02-25T09:45:44.877600Z","shell.execute_reply":"2024-02-25T09:45:44.885353Z"}}},{"cell_type":"code","source":"# help(audiomentations.Clip)\n\nClip = audiomentations.Clip(a_min=-0.1, a_max=0.1, p=1) \n\nres = Clip(waveform, sample_rate=44100)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:12.325774Z","iopub.execute_input":"2024-02-29T07:16:12.326261Z","iopub.status.idle":"2024-02-29T07:16:12.334335Z","shell.execute_reply.started":"2024-02-29T07:16:12.326214Z","shell.execute_reply":"2024-02-29T07:16:12.332605Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('waveform.max: {:.1f}'.format(waveform.max()))\nprint('waveform_after_clip.max: {:.1f}'.format(res.max()))","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:14.188171Z","iopub.execute_input":"2024-02-29T07:16:14.189762Z","iopub.status.idle":"2024-02-29T07:16:14.197856Z","shell.execute_reply.started":"2024-02-29T07:16:14.189689Z","shell.execute_reply":"2024-02-29T07:16:14.196081Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - Gain\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\nMultiply the audio by a random amplitude factor to reduce or increase the volume","metadata":{}},{"cell_type":"code","source":"Gain = audiomentations.Gain(min_gain_db=1, max_gain_db=5, p=1) \n\nres = Gain(waveform, 44100).max()\n\nprint('waveform.max: {:.1f}'.format(waveform.max()))\nprint('waveform_after_Gain.max: {:.1f}'.format(res.max()))","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:15.470205Z","iopub.execute_input":"2024-02-29T07:16:15.471129Z","iopub.status.idle":"2024-02-29T07:16:15.480794Z","shell.execute_reply.started":"2024-02-29T07:16:15.471078Z","shell.execute_reply":"2024-02-29T07:16:15.479313Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - Add_X_Noise","metadata":{}},{"cell_type":"markdown","source":"### - AddGaussianNoise\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\nAdd gaussian noise to the samples","metadata":{}},{"cell_type":"code","source":"# help(audiomentations.AddGaussianNoise)\n\nAddGaussianNoise = audiomentations.AddGaussianNoise(min_amplitude=0.001, # Minimum noise amplification factor\n max_amplitude=0.1, # Maximum noise amplification factor\n p=1)\n\nres = AddGaussianNoise(waveform, sample_rate=44100)\n\nprint('waveform.mean: {:.1f}'.format(waveform.max()))\nprint('waveform_after_AddGaussianNoise.mean: {:.1f}'.format(res.max()))","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:16.977763Z","iopub.execute_input":"2024-02-29T07:16:16.978516Z","iopub.status.idle":"2024-02-29T07:16:16.994721Z","shell.execute_reply.started":"2024-02-29T07:16:16.978480Z","shell.execute_reply":"2024-02-29T07:16:16.993220Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### - AddGaussianSNR\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\nAdd gaussian noise to the input. A random Signal to Noise Ratio (SNR) will be picked uniformly in the decibel scale. This aligns with human hearing, which is more logarithmic than linear.\n","metadata":{}},{"cell_type":"code","source":"# help(audiomentations.AddGaussianSNR) # min_snr_db max_snr_db\n\nAddGaussianSNR = audiomentations.AddGaussianSNR(min_snr_db=0.01, # Minimum signal-to-noise ratio in dB. A lower number means more noise.\n max_snr_db=0.1, # Maximum signal-to-noise ratio in dB. A greater number means less noise.\n p=1)\n\nres = AddGaussianSNR(waveform, sample_rate=44100).max()\n\nprint('waveform.mean: {:.1f}'.format(waveform.max()))\nprint('waveform_after_AddGaussianSNR.mean: {:.1f}'.format(res.max()))","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:20.191087Z","iopub.execute_input":"2024-02-29T07:16:20.191550Z","iopub.status.idle":"2024-02-29T07:16:20.208138Z","shell.execute_reply.started":"2024-02-29T07:16:20.191518Z","shell.execute_reply":"2024-02-29T07:16:20.207210Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### - AddColoredNoise\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\ncolored noise refers to noise that has specific spectral characteristics, rather than being flat or white noise, meaning that different frequencies exist with varying intensities. This may include pink noise, brown noise, or other noise types with specific frequency characteristics. Adding colored noise to audio can bring specific texture or characteristics to the sound and can be creatively used in audio processing and manipulation.","metadata":{}},{"cell_type":"code","source":"# help(torch_audiomentations.AddColoredNoise)\n\nAddColoredNoise = torch_audiomentations.AddColoredNoise(min_snr_in_db=3, # minimum SNR in dB.\n max_snr_in_db=30.0, # maximum SNR in dB.\n mode='per_example' # demo signal [cols, time, used to every single cols\n )\n\nres = AddColoredNoise(torch.tensor(waveform[None, ...][None, ...]), sample_rate=44100) # If your audio is mono, you can use a shape like [batch_size, 1, num_samples].","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:21.929887Z","iopub.execute_input":"2024-02-29T07:16:21.930492Z","iopub.status.idle":"2024-02-29T07:16:21.942759Z","shell.execute_reply.started":"2024-02-29T07:16:21.930448Z","shell.execute_reply":"2024-02-29T07:16:21.941460Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - HighPassFilter\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\n(May not be useful in the current competition as we do'nt know the frequency of the signal.😅)\n\n`HighPassFilter` is used to apply a high-pass filter to the input audio, removing low-frequency signals and preserving high-frequency signals. The function allows high-pass filtering of the audio based on parameterized filter slopes (6/12/18… dB/octave)","metadata":{}},{"cell_type":"code","source":"# help(audiomentations.HighPassFilter)\n\nsample_rate = 44100 \nduration = 5 \namplitude = 0.5 \n\nt = np.linspace(0, duration, int(sample_rate * duration), endpoint=False)\n\nfrequency1 = 240\nfrequency2 = 440\n\nwaveform_more_freq = amplitude * (np.sin(2 * np.pi * frequency1 * t) + np.sin(2 * np.pi * frequency2 * t))\n\naugment = audiomentations.HighPassFilter(min_cutoff_freq=430, \n min_rolloff=6, \n max_rolloff=18, \n p=1) # min_bandwidth_fraction, max_bandwidth_fraction\naugmented_waveform = augment(waveform_more_freq, sample_rate=sample_rate)\n\nstart = 100\nend = start + 1500\n\nplt.figure(figsize=(12, 6))\nplt.plot(t[start: end], waveform_more_freq[start: end], label='Original Signal')\nplt.plot(t[start: end], augmented_waveform[start: end], label='Filtered Signal')\nplt.xlabel('Time (s)')\nplt.ylabel('Amplitude')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:28.536647Z","iopub.execute_input":"2024-02-29T07:16:28.537106Z","iopub.status.idle":"2024-02-29T07:16:28.923921Z","shell.execute_reply.started":"2024-02-29T07:16:28.537074Z","shell.execute_reply":"2024-02-29T07:16:28.922827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_rate = 44100 \nduration = 5 \nfrequency = 440 \namplitude = 0.5 \n\nt = np.linspace(0, duration, int(sample_rate * duration), endpoint=False)\nwaveform = amplitude * np.sin(2 * np.pi * frequency * t) # ","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:31.944626Z","iopub.execute_input":"2024-02-29T07:16:31.945063Z","iopub.status.idle":"2024-02-29T07:16:31.959516Z","shell.execute_reply.started":"2024-02-29T07:16:31.945030Z","shell.execute_reply":"2024-02-29T07:16:31.958074Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(12, 6))\nplt.plot(t[start: end], waveform[start: end], label='waveform with only 440HZ')\nplt.plot(t[start: end], augmented_waveform[start: end], label='Filtered Signal')\nplt.xlabel('Time (s)')\nplt.ylabel('Amplitude')\nplt.title('Correctly filtered out signals with a frequency of 440Hz, but changed the amplitude')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:33.967392Z","iopub.execute_input":"2024-02-29T07:16:33.968516Z","iopub.status.idle":"2024-02-29T07:16:34.380273Z","shell.execute_reply.started":"2024-02-29T07:16:33.968462Z","shell.execute_reply":"2024-02-29T07:16:34.378722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(12, 6))\nplt.plot(t[start: end], augmented_waveform[start: end], label='Filtered Signal')\nplt.xlabel('Time (s)')\nplt.ylabel('Amplitude')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:37.259725Z","iopub.execute_input":"2024-02-29T07:16:37.261387Z","iopub.status.idle":"2024-02-29T07:16:37.591759Z","shell.execute_reply.started":"2024-02-29T07:16:37.261334Z","shell.execute_reply":"2024-02-29T07:16:37.590308Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - LowPassFilter\n\nsimilar to HighPassFilter\n\n(May not be useful in the current competition as we do'nt know the frequency of the signal.😅)","metadata":{}},{"cell_type":"code","source":"# help(audiomentations.LowPassFilter)\n\nsample_rate = 44100 \nduration = 5 \namplitude = 0.5 \n\nt = np.linspace(0, duration, int(sample_rate * duration), endpoint=False)\n\nfrequency1 = 240\nfrequency2 = 440\n\nwaveform_more_freq = amplitude * (np.sin(2 * np.pi * frequency1 * t) + np.sin(2 * np.pi * frequency2 * t))\n\naugment = audiomentations.LowPassFilter(min_cutoff_freq=100, \n max_cutoff_freq=300,\n min_rolloff=12, \n max_rolloff=18, \n p=1) # min_bandwidth_fraction, max_bandwidth_fraction\naugmented_waveform = augment(waveform_more_freq, sample_rate=sample_rate)\n\nstart = 100\nend = start + 1500\n\nplt.figure(figsize=(12, 6))\nplt.plot(t[start: end], waveform_more_freq[start: end], label='Original Signal')\nplt.plot(t[start: end], augmented_waveform[start: end], label='Filtered Signal')\nplt.xlabel('Time (s)')\nplt.ylabel('Amplitude')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:41.422003Z","iopub.execute_input":"2024-02-29T07:16:41.422421Z","iopub.status.idle":"2024-02-29T07:16:41.810951Z","shell.execute_reply.started":"2024-02-29T07:16:41.422393Z","shell.execute_reply":"2024-02-29T07:16:41.809477Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_rate = 44100 \nduration = 5 \nfrequency = 240 \namplitude = 0.5 \n\nt = np.linspace(0, duration, int(sample_rate * duration), endpoint=False)\nwaveform = amplitude * np.sin(2 * np.pi * frequency * t) # ","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:44.170847Z","iopub.execute_input":"2024-02-29T07:16:44.171270Z","iopub.status.idle":"2024-02-29T07:16:44.186088Z","shell.execute_reply.started":"2024-02-29T07:16:44.171225Z","shell.execute_reply":"2024-02-29T07:16:44.184270Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(12, 6))\nplt.plot(t[start: end], waveform[start: end], label='waveform with only 240HZ')\nplt.plot(t[start: end], augmented_waveform[start: end], label='Filtered Signal')\nplt.xlabel('Time (s)')\nplt.ylabel('Amplitude')\nplt.title('Correctly filtered out signals with a frequency of 440Hz, but changed the amplitude')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:46.975789Z","iopub.execute_input":"2024-02-29T07:16:46.976273Z","iopub.status.idle":"2024-02-29T07:16:47.323954Z","shell.execute_reply.started":"2024-02-29T07:16:46.976225Z","shell.execute_reply":"2024-02-29T07:16:47.322522Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - AdjustDuration\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\nTrim or pad the audio to the specified length/duration in samples or seconds. If the input sound is longer than the target duration, pick a random offset and crop the sound to the target duration. If the input sound is shorter than the target duration, pad the sound so the duration matches the target duration.\n","metadata":{}},{"cell_type":"code","source":"AdjustDuration = audiomentations.AdjustDuration(duration_samples=220600, \n padding_mode='silence', # 'silence' \"wrap\" or \"reflect\".\n padding_position='end', \n p=1) \n\nres = AdjustDuration(waveform, sample_rate=44100)\n\nprint('len(waveform): ', len(waveform))\n\nprint('len(waveform_after_AdjustDuration): ', len(res))","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:50.435064Z","iopub.execute_input":"2024-02-29T07:16:50.435506Z","iopub.status.idle":"2024-02-29T07:16:50.445757Z","shell.execute_reply.started":"2024-02-29T07:16:50.435475Z","shell.execute_reply":"2024-02-29T07:16:50.443809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_waveform(waveform=res, sample_split=1000, plt_title='waveform_after_AdjustDuration', position='end')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:16:52.233801Z","iopub.execute_input":"2024-02-29T07:16:52.234323Z","iopub.status.idle":"2024-02-29T07:16:52.462984Z","shell.execute_reply.started":"2024-02-29T07:16:52.234280Z","shell.execute_reply":"2024-02-29T07:16:52.461682Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - PolarityInversion\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\nFlip the audio samples upside-down, reversing their polarity. In other words, multiply the waveform by -1, so negative values become positive, and vice versa.\n\nThe difference between `audiomentations.Reverse` and `audiomentations.PolarityInversion` is that the former is a reversal on the time level, while the latter is a reversal of the sign of the sample values.","metadata":{}},{"cell_type":"code","source":"# help(audiomentations.PolarityInversion)\n\nsample_rate = 44100\nduration = 0.01\namplitude = 0.5\n\nt = np.linspace(0, duration, int(sample_rate * duration), endpoint=False)\n\nwaveform = amplitude * np.sin(2 * np.pi * 440 * t)\n\naugment = audiomentations.PolarityInversion(p=1)\n\ninverted_waveform = augment(samples=waveform, sample_rate=sample_rate)\n\nplt.figure(figsize=(12, 6))\nplt.subplot(2, 1, 1)\nplt.plot(t, waveform, label='Original waveform')\nplt.xlabel('Time (s)')\nplt.ylabel('Amplitude')\nplt.legend()\n\nplt.subplot(2, 1, 2)\nplt.plot(t, inverted_waveform, label='Inverted waveform')\nplt.xlabel('Time (s)')\nplt.ylabel('Amplitude')\nplt.legend()\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:17:19.137457Z","iopub.execute_input":"2024-02-29T07:17:19.137907Z","iopub.status.idle":"2024-02-29T07:17:19.653559Z","shell.execute_reply.started":"2024-02-29T07:17:19.137875Z","shell.execute_reply":"2024-02-29T07:17:19.652431Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - HighShelfFilter\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\n(May not be useful in the current competition as we do'nt know the frequency of the signal.😅)\n\nA high shelf filter is a filter that either boosts (increases amplitude) or cuts (decreases amplitude) frequencies above a certain center frequency. This transform applies a high-shelf filter at a specific center frequency in hertz.\n\n","metadata":{}},{"cell_type":"code","source":"# help(audiomentations.HighShelfFilter)\n\nsample_rate = 44100\nduration = 5\namplitude = 0.5\n\nt = np.linspace(0, duration, int(sample_rate * duration), endpoint=False)\n\nfrequency1 = 320\nfrequency2 = 350\nfrequency3 = 440\n\nwaveform_more_freq = amplitude * (np.sin(2 * np.pi * frequency1 * t) + np.sin(2 * np.pi * frequency2 * t) + np.sin(2 * np.pi * frequency3 * t))\n\nhigh_shelf = audiomentations.HighShelfFilter(min_center_freq=420, \n max_center_freq=460, \n min_gain_db=-18.0 , # The magnitude of reduction\n max_gain_db=18.0, \n min_q=0.99, # proboblity to reduction \n max_q=0.01, \n p=1.0)\naugmented_waveform = high_shelf(waveform_more_freq, sample_rate=sample_rate)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:17:44.867913Z","iopub.execute_input":"2024-02-29T07:17:44.868348Z","iopub.status.idle":"2024-02-29T07:17:44.895163Z","shell.execute_reply.started":"2024-02-29T07:17:44.868316Z","shell.execute_reply":"2024-02-29T07:17:44.893686Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"start = 100\nend = start + 1500\n\nplt.figure(figsize=(12, 6))\nplt.plot(t[start:end], waveform_more_freq[start:end], label='Original Signal')\nplt.plot(t[start:end], augmented_waveform[start:end], label='Filtered Signal')\nplt.xlabel('Time (s)')\nplt.ylabel('Amplitude')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:17:46.464601Z","iopub.execute_input":"2024-02-29T07:17:46.465031Z","iopub.status.idle":"2024-02-29T07:17:46.768021Z","shell.execute_reply.started":"2024-02-29T07:17:46.465002Z","shell.execute_reply":"2024-02-29T07:17:46.766579Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - Resample\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\nResample signal using librosa.core.resample. To do downsampling only set both minimum and maximum sampling rate lower than original sampling rate and vice versa to do upsampling only.","metadata":{}},{"cell_type":"code","source":"# help(audiomentations.Resample)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:06:43.457194Z","iopub.status.idle":"2024-02-29T07:06:43.457799Z","shell.execute_reply.started":"2024-02-29T07:06:43.457503Z","shell.execute_reply":"2024-02-29T07:06:43.457527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_rate = 44100 \nduration = 5 \nfrequency = 440 \namplitude = 0.5 \n\nt = np.linspace(0, duration, int(sample_rate * duration), endpoint=False)\nwaveform = amplitude * np.sin(2 * np.pi * frequency * t) ","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:17:51.243238Z","iopub.execute_input":"2024-02-29T07:17:51.243702Z","iopub.status.idle":"2024-02-29T07:17:51.256610Z","shell.execute_reply.started":"2024-02-29T07:17:51.243655Z","shell.execute_reply":"2024-02-29T07:17:51.254973Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Resample = audiomentations.Resample(min_sample_rate=800, # Minimum sample rate\n max_sample_rate=1000, # Maximum sample rate\n p=1) #\n\nres = Resample(waveform, sample_rate=44100)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:17:54.192949Z","iopub.execute_input":"2024-02-29T07:17:54.193469Z","iopub.status.idle":"2024-02-29T07:17:54.206634Z","shell.execute_reply.started":"2024-02-29T07:17:54.193425Z","shell.execute_reply":"2024-02-29T07:17:54.205138Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('len of wavefrom: ', len(waveform))\n\nprint('len of waveform_after_resample: ', len(res))","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:17:56.272071Z","iopub.execute_input":"2024-02-29T07:17:56.272596Z","iopub.status.idle":"2024-02-29T07:17:56.281436Z","shell.execute_reply.started":"2024-02-29T07:17:56.272558Z","shell.execute_reply":"2024-02-29T07:17:56.280015Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_waveform(waveform=res, sample_split=250, plt_title='waveform_after_resample', position='start')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:18:09.509883Z","iopub.execute_input":"2024-02-29T07:18:09.510299Z","iopub.status.idle":"2024-02-29T07:18:09.770075Z","shell.execute_reply.started":"2024-02-29T07:18:09.510269Z","shell.execute_reply":"2024-02-29T07:18:09.768780Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - Reverse\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\nThe difference between `audiomentations.Reverse` and `audiomentations.PolarityInversion` is that the former is a time-domain reversal, while the latter is a reversal of the sample amplitude sign","metadata":{}},{"cell_type":"code","source":"Reverse = audiomentations.Reverse(p=1)\n\nres = Reverse(waveform, sample_rate=44100)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:18:18.180219Z","iopub.execute_input":"2024-02-29T07:18:18.180714Z","iopub.status.idle":"2024-02-29T07:18:18.188116Z","shell.execute_reply.started":"2024-02-29T07:18:18.180682Z","shell.execute_reply":"2024-02-29T07:18:18.186275Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_waveform(waveform=waveform, sample_split=1000, plt_title='waveform', position='start')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:18:19.502031Z","iopub.execute_input":"2024-02-29T07:18:19.502511Z","iopub.status.idle":"2024-02-29T07:18:19.742948Z","shell.execute_reply.started":"2024-02-29T07:18:19.502479Z","shell.execute_reply":"2024-02-29T07:18:19.741591Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_waveform(waveform=res, sample_split=1000, plt_title='waveform_after_Reverse', position='start')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:18:22.061680Z","iopub.execute_input":"2024-02-29T07:18:22.062164Z","iopub.status.idle":"2024-02-29T07:18:22.296463Z","shell.execute_reply.started":"2024-02-29T07:18:22.062129Z","shell.execute_reply":"2024-02-29T07:18:22.294981Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - Normalize\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\nApply a constant amount of gain, so that highest signal level present in the sound becomes 0 dBFS, i.e. the loudest level allowed if all samples must be between -1 and 1. Also known as peak normalization.","metadata":{}},{"cell_type":"code","source":"# help(audiomentations.Normalize)\n\nNormalize = audiomentations.Normalize(apply_to='all', p=1)\n\nres = Normalize(waveform, sample_rate=44100)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:18:26.573219Z","iopub.execute_input":"2024-02-29T07:18:26.573769Z","iopub.status.idle":"2024-02-29T07:18:26.582895Z","shell.execute_reply.started":"2024-02-29T07:18:26.573733Z","shell.execute_reply":"2024-02-29T07:18:26.581121Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_waveform(waveform=res, sample_split=1000, plt_title='waveform_after_Normalize', position='start')","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:18:29.127388Z","iopub.execute_input":"2024-02-29T07:18:29.128768Z","iopub.status.idle":"2024-02-29T07:18:29.397087Z","shell.execute_reply.started":"2024-02-29T07:18:29.128713Z","shell.execute_reply":"2024-02-29T07:18:29.395552Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - BandPassFilter\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\n(May not be useful in the current competition as we do'nt know the frequency of the signal.😅)\n\n`audiomentations.BandPassFilter` function is used to apply a band-pass filtering operation to the input audio. Band-pass filtering is a signal processing technique that allows only a certain range of frequencies to pass through while suppressing other frequency signals. ","metadata":{}},{"cell_type":"code","source":"# help(audiomentations.BandStopFilter)\n\nsample_rate = 44100 \nduration = 5 \namplitude = 0.5 \n\nt = np.linspace(0, duration, int(sample_rate * duration), endpoint=False)\n\nfrequency1 = 140\nfrequency2 = 440\n\n\nwaveform_with_freq = amplitude * (np.sin(2 * np.pi * frequency1 * t) + np.sin(2 * np.pi * frequency2 * t))\n\n# Min_center_freq and max_center_freq represent the center frequency range of the bandpass filter. The filter will only allow signals within this frequency range to pass through\naugment = audiomentations.BandPassFilter(min_center_freq=420, \n max_center_freq=450,\n min_bandwidth_fraction=0.5, \n max_bandwidth_fraction=1, \n p=1) # min_bandwidth_fraction, max_bandwidth_fraction\naugmented_waveform = augment(waveform_with_freq, sample_rate=sample_rate)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:18:32.537036Z","iopub.execute_input":"2024-02-29T07:18:32.538673Z","iopub.status.idle":"2024-02-29T07:18:32.565913Z","shell.execute_reply.started":"2024-02-29T07:18:32.538589Z","shell.execute_reply":"2024-02-29T07:18:32.563688Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"start = 1000\nend = start+1000\n\nplt.figure(figsize=(12, 6))\nplt.plot(t[start:end], waveform_with_freq[start:end], label='Original Signal')\nplt.plot(t[start:end], augmented_waveform[start:end], label='Filtered Signal')\nplt.xlabel('Time (s)')\nplt.ylabel('Amplitude')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:18:37.379415Z","iopub.execute_input":"2024-02-29T07:18:37.379852Z","iopub.status.idle":"2024-02-29T07:18:37.720955Z","shell.execute_reply.started":"2024-02-29T07:18:37.379822Z","shell.execute_reply":"2024-02-29T07:18:37.719501Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## - BandStopFilter\n\n- Type of input data: **time domain**\n- Function's role: DataAugment\n\n(May not be useful in the current competition as we do'nt know the frequency of the signal.😅)\n\nsimilar to BandpassFilter","metadata":{}},{"cell_type":"code","source":"# help(audiomentations.BandStopFilter)\n\nsample_rate = 44100 \nduration = 5 \namplitude = 0.5 \n\nt = np.linspace(0, duration, int(sample_rate * duration), endpoint=False)\n\nfrequency1 = 320\nfrequency2 = 350\nfrequency3 = 440\n\nwaveform_more_freq = amplitude * (np.sin(2 * np.pi * frequency1 * t) + np.sin(2 * np.pi * frequency2 * t) + np.sin(2 * np.pi * frequency3 * t))\n\naugment = audiomentations.BandStopFilter(min_center_freq=315, max_center_freq=355, p=1) # min_bandwidth_fraction, max_bandwidth_fraction\naugmented_waveform = augment(waveform_more_freq, sample_rate=sample_rate)","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:18:41.402491Z","iopub.execute_input":"2024-02-29T07:18:41.402901Z","iopub.status.idle":"2024-02-29T07:18:41.432808Z","shell.execute_reply.started":"2024-02-29T07:18:41.402872Z","shell.execute_reply":"2024-02-29T07:18:41.431600Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"start = 0\nend = start+1500\n\nplt.figure(figsize=(12, 6))\nplt.plot(t[start:end], waveform_more_freq[start:end], label='Original Signal')\nplt.plot(t[start:end], augmented_waveform[start:end], label='Filtered Signal')\nplt.xlabel('Time (s)')\nplt.ylabel('Amplitude')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-29T07:18:43.448382Z","iopub.execute_input":"2024-02-29T07:18:43.448797Z","iopub.status.idle":"2024-02-29T07:18:43.812566Z","shell.execute_reply.started":"2024-02-29T07:18:43.448766Z","shell.execute_reply":"2024-02-29T07:18:43.811107Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Thanks for watching!","metadata":{}},{"cell_type":"markdown","source":"**If you find this notebook useful, please upvoted, thank you!** ⭐️⭐️⭐️⭐️⭐️","metadata":{}}]}