\nThis starter notebook is provided by the Keras team.
","metadata":{}},{"cell_type":"markdown","source":"# BirdCLEF 2024 with [KerasCV](https://github.com/keras-team/keras-cv) and [Keras](https://github.com/keras-team/keras)\n\n> The objective of this competition is to identify under-studied Indian bird species by their calls.\n\n
\n \n
\n\nThis notebook guides you through the process of training a Deep Learning model to recognize bird species by their songs (audio data). Specifically, this notebook uses the EfficientNetV2 backbone from KerasCV on the competition dataset. It also shows how to convert audio data to mel-spectrograms using Keras.\n\nSince this competition requires inference to be run only on CPU, we need to prepare separate notebooks for training (to run on GPU) and inference (to run on CPU). You can find the [inference notebook here](https://www.kaggle.com/code/awsaf49/birdclef24-kerascv-starter-infer).\n\nFun fact: This notebook is backend-agnostic, supporting TensorFlow, PyTorch, and JAX. Utilizing KerasCV and Keras allows us to choose our preferred backend. Explore more details on [Keras](https://keras.io/keras_core/announcement/).\n\nIn this notebook, you will learn:\n\n- Designing a data pipeline for audio data, including audio-to-spectrogram conversion.\n- Creating an augmentation pipeline with KerasCV.\n- Loading the data efficiently using [`tf.data`](https://www.tensorflow.org/guide/data).\n- Creating the model using KerasCV presets.\n- Training the model.\n\n**Note**: For a more in-depth understanding of KerasCV, refer to the [KerasCV guides](https://keras.io/guides/keras_cv/).\n","metadata":{}},{"cell_type":"markdown","source":"# Import Libraries 📚","metadata":{"papermill":{"duration":0.065343,"end_time":"2022-03-08T03:18:11.885586","exception":false,"start_time":"2022-03-08T03:18:11.820243","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import os\nos.environ[\"KERAS_BACKEND\"] = \"jax\" # \"jax\" or \"tensorflow\" or \"torch\" \n\nimport keras_cv\nimport keras\nimport keras.backend as K\nimport tensorflow as tf\nimport tensorflow_io as tfio\n\nimport numpy as np \nimport pandas as pd\n\nfrom glob import glob\nfrom tqdm import tqdm\n\nimport librosa\nimport IPython.display as ipd\nimport librosa.display as lid\n\nimport matplotlib.pyplot as plt\nimport matplotlib as mpl\n\ncmap = mpl.cm.get_cmap('coolwarm')","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","papermill":{"duration":2.632068,"end_time":"2022-03-08T03:18:14.585094","exception":false,"start_time":"2022-03-08T03:18:11.953026","status":"completed"},"tags":[],"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-06-05T07:06:08.150990Z","iopub.execute_input":"2024-06-05T07:06:08.151286Z","iopub.status.idle":"2024-06-05T07:06:30.450009Z","shell.execute_reply.started":"2024-06-05T07:06:08.151260Z","shell.execute_reply":"2024-06-05T07:06:30.448993Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Library Version","metadata":{"papermill":{"duration":0.065649,"end_time":"2022-03-08T03:18:14.717311","exception":false,"start_time":"2022-03-08T03:18:14.651662","status":"completed"},"tags":[]}},{"cell_type":"code","source":"print(\"TensorFlow:\", tf.__version__)\nprint(\"Keras:\", keras.__version__)\nprint(\"KerasCV:\", keras_cv.__version__)","metadata":{"papermill":{"duration":0.155095,"end_time":"2022-03-08T03:18:14.939054","exception":false,"start_time":"2022-03-08T03:18:14.783959","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-05T07:06:55.721409Z","iopub.execute_input":"2024-06-05T07:06:55.721791Z","iopub.status.idle":"2024-06-05T07:06:55.727281Z","shell.execute_reply.started":"2024-06-05T07:06:55.721749Z","shell.execute_reply":"2024-06-05T07:06:55.726231Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Configuration ⚙️","metadata":{"papermill":{"duration":0.066353,"end_time":"2022-03-08T03:18:18.099835","exception":false,"start_time":"2022-03-08T03:18:18.033482","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class CFG:\n seed = 42\n \n # Input image size and batch size\n img_size = [128, 384]\n batch_size = 64\n \n # Audio duration, sample rate, and length\n duration = 15 # second\n sample_rate = 32000\n audio_len = duration*sample_rate\n \n # STFT parameters\n nfft = 2028\n window = 2048\n hop_length = audio_len // (img_size[1] - 1)\n fmin = 20\n fmax = 16000\n \n # Number of epochs, model name\n epochs = 10\n preset = 'VGG16_imagenet'\n \n # Data augmentation parameters\n augment=True\n\n # Class Labels for BirdCLEF 24\n class_names = sorted(os.listdir('/kaggle/input/birdclef-2024/train_audio/'))\n num_classes = len(class_names)\n class_labels = list(range(num_classes))\n label2name = dict(zip(class_labels, class_names))\n name2label = {v:k for k,v in label2name.items()}\n \n","metadata":{"papermill":{"duration":0.156464,"end_time":"2022-03-08T03:18:18.322809","exception":false,"start_time":"2022-03-08T03:18:18.166345","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-05T07:06:59.186385Z","iopub.execute_input":"2024-06-05T07:06:59.187294Z","iopub.status.idle":"2024-06-05T07:06:59.204184Z","shell.execute_reply.started":"2024-06-05T07:06:59.187255Z","shell.execute_reply":"2024-06-05T07:06:59.203427Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Reproducibility ♻️\nSets value for random seed to produce similar result in each run.","metadata":{"papermill":{"duration":0.070351,"end_time":"2022-03-08T03:18:18.46058","exception":false,"start_time":"2022-03-08T03:18:18.390229","status":"completed"},"tags":[]}},{"cell_type":"code","source":"tf.keras.utils.set_random_seed(CFG.seed)","metadata":{"papermill":{"duration":0.153451,"end_time":"2022-03-08T03:18:18.685056","exception":false,"start_time":"2022-03-08T03:18:18.531605","status":"completed"},"tags":[],"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-06-05T07:07:03.368492Z","iopub.execute_input":"2024-06-05T07:07:03.369450Z","iopub.status.idle":"2024-06-05T07:07:03.373700Z","shell.execute_reply.started":"2024-06-05T07:07:03.369418Z","shell.execute_reply":"2024-06-05T07:07:03.372775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Dataset Path 📁","metadata":{}},{"cell_type":"code","source":"BASE_PATH = '/kaggle/input/birdclef-2024'","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-06-05T07:07:06.548361Z","iopub.execute_input":"2024-06-05T07:07:06.548940Z","iopub.status.idle":"2024-06-05T07:07:06.552880Z","shell.execute_reply.started":"2024-06-05T07:07:06.548903Z","shell.execute_reply":"2024-06-05T07:07:06.551942Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Meta Data 📖","metadata":{"papermill":{"duration":0.067107,"end_time":"2022-03-08T03:18:26.962626","exception":false,"start_time":"2022-03-08T03:18:26.895519","status":"completed"},"tags":[]}},{"cell_type":"code","source":"df = pd.read_csv(f'{BASE_PATH}/train_metadata.csv')\ndf['filepath'] = BASE_PATH + '/train_audio/' + df.filename\ndf['target'] = df.primary_label.map(CFG.name2label)\ndf['filename'] = df.filepath.map(lambda x: x.split('/')[-1])\ndf['xc_id'] = df.filepath.map(lambda x: x.split('/')[-1].split('.')[0])\n\n# Display rwos\ndf.head(2)","metadata":{"papermill":{"duration":0.241649,"end_time":"2022-03-08T03:18:27.408813","exception":false,"start_time":"2022-03-08T03:18:27.167164","status":"completed"},"tags":[],"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-06-05T07:07:09.460421Z","iopub.execute_input":"2024-06-05T07:07:09.461028Z","iopub.status.idle":"2024-06-05T07:07:09.724566Z","shell.execute_reply.started":"2024-06-05T07:07:09.460996Z","shell.execute_reply":"2024-06-05T07:07:09.723634Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# EDA 🎨","metadata":{}},{"cell_type":"markdown","source":"## Utility","metadata":{}},{"cell_type":"code","source":"def load_audio(filepath):\n audio, sr = librosa.load(filepath)\n return audio, sr\n\ndef get_spectrogram(audio):\n spec = librosa.feature.melspectrogram(y=audio, \n sr=CFG.sample_rate, \n n_mels=256,\n n_fft=2048,\n hop_length=512,\n fmax=CFG.fmax,\n fmin=CFG.fmin,\n )\n spec = librosa.power_to_db(spec, ref=1.0)\n min_ = spec.min()\n max_ = spec.max()\n if max_ != min_:\n spec = (spec - min_)/(max_ - min_)\n return spec\n\ndef display_audio(row):\n # Caption for viz\n caption = f'Id: {row.filename} | Name: {row.common_name} | Sci.Name: {row.scientific_name} | Rating: {row.rating}'\n # Read audio file\n audio, sr = load_audio(row.filepath)\n # Keep fixed length audio\n audio = audio[:CFG.audio_len]\n # Spectrogram from audio\n spec = get_spectrogram(audio)\n # Display audio\n print(\"# Audio:\")\n display(ipd.Audio(audio, rate=CFG.sample_rate))\n print('# Visualization:')\n fig, ax = plt.subplots(2, 1, figsize=(12, 2*3), sharex=True, tight_layout=True)\n fig.suptitle(caption)\n # Waveplot\n lid.waveshow(audio,\n sr=CFG.sample_rate,\n ax=ax[0],\n color= cmap(0.1))\n # Specplot\n lid.specshow(spec, \n sr = CFG.sample_rate, \n hop_length=512,\n n_fft=2048,\n fmin=CFG.fmin,\n fmax=CFG.fmax,\n x_axis = 'time', \n y_axis = 'mel',\n cmap = 'coolwarm',\n ax=ax[1])\n ax[0].set_xlabel('');\n fig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-06-05T07:07:13.822266Z","iopub.execute_input":"2024-06-05T07:07:13.823014Z","iopub.status.idle":"2024-06-05T07:07:13.833965Z","shell.execute_reply.started":"2024-06-05T07:07:13.822980Z","shell.execute_reply":"2024-06-05T07:07:13.832931Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Sample 1","metadata":{}},{"cell_type":"code","source":"row = df.iloc[35]\n\n# Display audio\ndisplay_audio(row)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-06-05T07:07:24.639514Z","iopub.execute_input":"2024-06-05T07:07:24.640341Z","iopub.status.idle":"2024-06-05T07:07:41.458277Z","shell.execute_reply.started":"2024-06-05T07:07:24.640294Z","shell.execute_reply":"2024-06-05T07:07:41.457373Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Sample 2","metadata":{}},{"cell_type":"code","source":"row = df.iloc[16]\n\n# Display audio\ndisplay_audio(row)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-06-05T07:07:41.459635Z","iopub.execute_input":"2024-06-05T07:07:41.460201Z","iopub.status.idle":"2024-06-05T07:07:43.302549Z","shell.execute_reply.started":"2024-06-05T07:07:41.460174Z","shell.execute_reply":"2024-06-05T07:07:43.301538Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Sample 3","metadata":{}},{"cell_type":"code","source":"row = df.iloc[50]\n\n# Display audio\ndisplay_audio(row)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-06-05T07:07:47.156706Z","iopub.execute_input":"2024-06-05T07:07:47.157099Z","iopub.status.idle":"2024-06-05T07:07:48.731622Z","shell.execute_reply.started":"2024-06-05T07:07:47.157070Z","shell.execute_reply":"2024-06-05T07:07:48.730767Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Split 🔪\nFollowing code will split the data into folds using target stratification.\n> **Note:** Some classess have too few samples thus not each fold contains all the classes. ","metadata":{"papermill":{"duration":0.09524,"end_time":"2022-03-08T03:18:34.861029","exception":false,"start_time":"2022-03-08T03:18:34.765789","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Import required packages\nfrom sklearn.model_selection import train_test_split\n\ntrain_df, valid_df = train_test_split(df, test_size=0.2)\n\nprint(f\"Num Train: {len(train_df)} | Num Valid: {len(valid_df)}\")","metadata":{"papermill":{"duration":0.386301,"end_time":"2022-03-08T03:18:35.325064","exception":false,"start_time":"2022-03-08T03:18:34.938763","status":"completed"},"tags":[],"_kg_hide-input":false,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-06-05T07:07:52.177775Z","iopub.execute_input":"2024-06-05T07:07:52.178162Z","iopub.status.idle":"2024-06-05T07:07:52.362058Z","shell.execute_reply.started":"2024-06-05T07:07:52.178127Z","shell.execute_reply":"2024-06-05T07:07:52.360991Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Loader 🍚","metadata":{"papermill":{"duration":0.152812,"end_time":"2022-03-08T03:18:48.676686","exception":false,"start_time":"2022-03-08T03:18:48.523874","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"## Decoders\n\nThe following code will decode the raw audio from `.ogg` file and also decode the spectrogram from the `audio` file. Additionally, we will apply Z-Score standardization and Min-Max normalization to ensure consistent inputs to the model.\n","metadata":{}},{"cell_type":"code","source":"# Decodes Audio\ndef build_decoder(with_labels=True, dim=1024):\n def get_audio(filepath):\n file_bytes = tf.io.read_file(filepath)\n audio = tfio.audio.decode_vorbis(file_bytes) # decode .ogg file\n audio = tf.cast(audio, tf.float32)\n if tf.shape(audio)[1] > 1: # stereo -> mono\n audio = audio[..., 0:1]\n audio = tf.squeeze(audio, axis=-1)\n return audio\n\n def crop_or_pad(audio, target_len, pad_mode=\"constant\"):\n audio_len = tf.shape(audio)[0]\n diff_len = abs(\n target_len - audio_len\n ) # find difference between target and audio length\n if audio_len < target_len: # do padding if audio length is shorter\n pad1 = tf.random.uniform([], maxval=diff_len, dtype=tf.int32)\n pad2 = diff_len - pad1\n audio = tf.pad(audio, paddings=[[pad1, pad2]], mode=pad_mode)\n elif audio_len > target_len: # do cropping if audio length is larger\n idx = tf.random.uniform([], maxval=diff_len, dtype=tf.int32)\n audio = audio[idx : (idx + target_len)]\n return tf.reshape(audio, [target_len])\n\n def apply_preproc(spec):\n # Standardize\n mean = tf.math.reduce_mean(spec)\n std = tf.math.reduce_std(spec)\n spec = tf.where(tf.math.equal(std, 0), spec - mean, (spec - mean) / std)\n\n # Normalize using Min-Max\n min_val = tf.math.reduce_min(spec)\n max_val = tf.math.reduce_max(spec)\n spec = tf.where(\n tf.math.equal(max_val - min_val, 0),\n spec - min_val,\n (spec - min_val) / (max_val - min_val),\n )\n return spec\n\n def get_target(target):\n target = tf.reshape(target, [1])\n target = tf.cast(tf.one_hot(target, CFG.num_classes), tf.float32)\n target = tf.reshape(target, [CFG.num_classes])\n return target\n\n def decode(path):\n # Load audio file\n audio = get_audio(path)\n # Crop or pad audio to keep a fixed length\n audio = crop_or_pad(audio, dim)\n # Audio to Spectrogram\n spec = keras.layers.MelSpectrogram(\n num_mel_bins=CFG.img_size[0],\n fft_length=CFG.nfft,\n sequence_stride=CFG.hop_length,\n sampling_rate=CFG.sample_rate,\n )(audio)\n # Apply normalization and standardization\n spec = apply_preproc(spec)\n # Spectrogram to 3 channel image (for imagenet)\n spec = tf.tile(spec[..., None], [1, 1, 3])\n spec = tf.reshape(spec, [*CFG.img_size, 3])\n return spec\n\n def decode_with_labels(path, label):\n label = get_target(label)\n return decode(path), label\n\n return decode_with_labels if with_labels else decode\n","metadata":{"papermill":{"duration":0.251237,"end_time":"2022-03-08T03:18:49.079346","exception":false,"start_time":"2022-03-08T03:18:48.828109","status":"completed"},"tags":[],"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-06-05T07:07:56.721099Z","iopub.execute_input":"2024-06-05T07:07:56.721467Z","iopub.status.idle":"2024-06-05T07:07:56.739006Z","shell.execute_reply.started":"2024-06-05T07:07:56.721437Z","shell.execute_reply":"2024-06-05T07:07:56.737915Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Augmenters\nFollowing code will apply augmentations to spectrogram data. In this notebook, we will use MixUp, CutOut (TimeMasking and FreqMasking) from KerasCV.\n\n> Note that, these augmentations will be applied to batch of spectrograms rather than single spectrograms.","metadata":{"papermill":{"duration":0.150182,"end_time":"2022-03-08T03:18:49.38214","exception":false,"start_time":"2022-03-08T03:18:49.231958","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def build_augmenter():\n augmenters = [\n keras_cv.layers.MixUp(alpha=0.4),\n keras_cv.layers.RandomCutout(height_factor=(1.0, 1.0),\n width_factor=(0.06, 0.12)), # time-masking\n keras_cv.layers.RandomCutout(height_factor=(0.06, 0.1),\n width_factor=(1.0, 1.0)), # freq-masking\n ]\n \n def augment(img, label):\n data = {\"images\":img, \"labels\":label}\n for augmenter in augmenters:\n if tf.random.uniform([]) < 0.35:\n data = augmenter(data, training=True)\n return data[\"images\"], data[\"labels\"]\n \n return augment","metadata":{"execution":{"iopub.status.busy":"2024-06-05T07:08:03.631316Z","iopub.execute_input":"2024-06-05T07:08:03.632018Z","iopub.status.idle":"2024-06-05T07:08:03.638773Z","shell.execute_reply.started":"2024-06-05T07:08:03.631986Z","shell.execute_reply":"2024-06-05T07:08:03.637784Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data Pipeline\nFollowing code builds the complete pipeline of the data flow. It uses `tf.data.Dataset` for data processing. Here are some cool features of `tf.data`,\n* We can build complex input pipelines from simple, reusable pieces using`tf.data` API . For example, the pipeline for an audio model might aggregate data from files in a distributed file system, apply random transformation/augmentation to each audio/spectrogram, and merge randomly selected data into a batch for training.\n* Moreover `tf.data` API provides a `tf.data.Dataset` feature that represents a sequence of components where each component comprises one or more pieces. For instance, in an audio pipeline, a component might be a single training example, with a pair of tensor pieces representing the audio and its label.\n\nCheck out this [doc](https://www.tensorflow.org/guide/data) if you want to learn more about `tf.data`.","metadata":{"papermill":{"duration":0.152217,"end_time":"2022-03-08T03:18:50.097623","exception":false,"start_time":"2022-03-08T03:18:49.945406","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def build_dataset(paths, labels=None, batch_size=32, \n decode_fn=None, augment_fn=None, cache=True,\n augment=False, shuffle=2048):\n\n if decode_fn is None:\n decode_fn = build_decoder(labels is not None, dim=CFG.audio_len)\n\n if augment_fn is None:\n augment_fn = build_augmenter()\n \n AUTO = tf.data.experimental.AUTOTUNE\n slices = (paths,) if labels is None else (paths, labels)\n ds = tf.data.Dataset.from_tensor_slices(slices)\n ds = ds.map(decode_fn, num_parallel_calls=AUTO)\n ds = ds.cache() if cache else ds\n if shuffle:\n opt = tf.data.Options()\n ds = ds.shuffle(shuffle, seed=CFG.seed)\n opt.experimental_deterministic = False\n ds = ds.with_options(opt)\n ds = ds.batch(batch_size, drop_remainder=True)\n ds = ds.map(augment_fn, num_parallel_calls=AUTO) if augment else ds\n ds = ds.prefetch(AUTO)\n return ds","metadata":{"papermill":{"duration":0.240881,"end_time":"2022-03-08T03:18:50.489717","exception":false,"start_time":"2022-03-08T03:18:50.248836","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-05T07:08:07.455864Z","iopub.execute_input":"2024-06-05T07:08:07.456817Z","iopub.status.idle":"2024-06-05T07:08:07.465513Z","shell.execute_reply.started":"2024-06-05T07:08:07.456778Z","shell.execute_reply":"2024-06-05T07:08:07.464422Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Build Train and Valid Dataloaders","metadata":{}},{"cell_type":"code","source":"# Train\ntrain_paths = train_df.filepath.values\ntrain_labels = train_df.target.values\ntrain_ds = build_dataset(train_paths, train_labels, batch_size=CFG.batch_size,\n shuffle=True, augment=CFG.augment)\n\n# Valid\nvalid_paths = valid_df.filepath.values\nvalid_labels = valid_df.target.values\nvalid_ds = build_dataset(valid_paths, valid_labels, batch_size=CFG.batch_size,\n shuffle=False, augment=False)","metadata":{"execution":{"iopub.status.busy":"2024-06-05T07:08:11.073657Z","iopub.execute_input":"2024-06-05T07:08:11.074026Z","iopub.status.idle":"2024-06-05T07:08:15.639883Z","shell.execute_reply.started":"2024-06-05T07:08:11.073995Z","shell.execute_reply":"2024-06-05T07:08:15.638898Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Visualization 🔭\nTo ensure our pipeline is generating **spectrogram** and its associate **label** correctly, we'll check some samples from a batch.","metadata":{}},{"cell_type":"code","source":"def plot_batch(batch, row=3, col=3, label2name=None,):\n \"\"\"Plot one batch data\"\"\"\n if isinstance(batch, tuple) or isinstance(batch, list):\n specs, tars = batch\n else:\n specs = batch\n tars = None\n plt.figure(figsize=(col*5, row*3))\n for idx in range(row*col):\n ax = plt.subplot(row, col, idx+1)\n lid.specshow(np.array(specs[idx, ..., 0]), \n n_fft=CFG.nfft, \n hop_length=CFG.hop_length, \n sr=CFG.sample_rate,\n x_axis='time',\n y_axis='mel',\n cmap='coolwarm')\n if tars is not None:\n label = tars[idx].numpy().argmax()\n name = label2name[label]\n plt.title(name)\n plt.tight_layout()\n plt.show()","metadata":{"papermill":{"duration":0.328513,"end_time":"2022-03-08T03:19:59.512224","exception":false,"start_time":"2022-03-08T03:19:59.183711","status":"completed"},"tags":[],"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-06-05T07:08:21.953581Z","iopub.execute_input":"2024-06-05T07:08:21.954519Z","iopub.status.idle":"2024-06-05T07:08:21.962309Z","shell.execute_reply.started":"2024-06-05T07:08:21.954485Z","shell.execute_reply":"2024-06-05T07:08:21.961253Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_ds = train_ds.take(100)\nbatch = next(iter(sample_ds))\nplot_batch(batch, label2name=CFG.label2name)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-06-05T07:08:26.173388Z","iopub.execute_input":"2024-06-05T07:08:26.173766Z","iopub.status.idle":"2024-06-05T07:08:38.818705Z","shell.execute_reply.started":"2024-06-05T07:08:26.173734Z","shell.execute_reply":"2024-06-05T07:08:38.817674Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🤖 Modeling\n\nBuilding a model for an audio recognition task with spectrograms as input is quite straightforward, as it is very similar to image classification. This is because the shape of spectrogram data is very similar to image data. In this notebook, to perform the audio recognition task, we will utilize the `EfficientNetV2` ImageNet-pretrained model as the backbone. Even though this backbone is pretrained with ImageNet data instead of spectrogram data, we can leverage transfer learning to adapt it to our spectrogram-based task.\n\n> Note that we can train our model on any duration audio file (here we are using `10 seconds`), but we will always infer on `5-second` audio files (as per competition rules). To facilitate this, we have set the model input shape to `(None, None, 3)`, which will allow us to have variable-length input during training and inference.\n\n\nIn case you are wondering, **Why not train and infer on both `5-second`?** In the train data, we have long audio files, but we are not sure which part of the audio contains the labeled bird's song. In other words, this is weakly labeled. To ensure the provided label is accurately suited to the audio, we are using a larger audio size than `5 seconds`. You are welcome to try out different audio lengths for training.\n","metadata":{"papermill":{"duration":0.182769,"end_time":"2022-03-08T03:20:04.861966","exception":false,"start_time":"2022-03-08T03:20:04.679197","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Create an input layer for the model\ninp = keras.layers.Input(shape=(None, None, 3))\nfrom tensorflow.keras.applications import VGG16\n\n # Load VGG16 model with pre-trained ImageNet weights\nbackbone = VGG16(weights='imagenet', include_top=False)\n\nout = keras_cv.models.ImageClassifier(\n backbone=backbone,\n num_classes=CFG.num_classes,\n name=\"classifier\"\n)(inp)\n# Build model\nmodel = keras.models.Model(inputs=inp, outputs=out)\n# Compile model with optimizer, loss and metrics\nmodel.compile(optimizer=\"adam\",\n loss=keras.losses.CategoricalCrossentropy(label_smoothing=0.02),\n metrics=[keras.metrics.AUC(name='auc')],\n )\nmodel.summary()","metadata":{"papermill":{"duration":1.239321,"end_time":"2022-03-08T03:20:06.281118","exception":false,"start_time":"2022-03-08T03:20:05.041797","status":"completed"},"tags":[],"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-06-05T07:08:57.329975Z","iopub.execute_input":"2024-06-05T07:08:57.330324Z","iopub.status.idle":"2024-06-05T07:09:00.227941Z","shell.execute_reply.started":"2024-06-05T07:08:57.330296Z","shell.execute_reply":"2024-06-05T07:09:00.227015Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# LR Schedule ⚓\n* Learning Rate scheduler for transfer learning. \n* The learning rate starts from `lr_start`, then decreases to a`lr_min` using different methods namely,\n * **step**: Reduce lr step wise like stair.\n * **cos**: Follow Cosine graph to reduce lr.\n * **exp**: Reduce lr exponentially.","metadata":{}},{"cell_type":"code","source":"import math\n\ndef get_lr_callback(batch_size=8, mode='cos', epochs=10, plot=False):\n lr_start, lr_max, lr_min = 5e-5, 8e-6 * batch_size, 1e-5\n lr_ramp_ep, lr_sus_ep, lr_decay = 3, 0, 0.75\n\n def lrfn(epoch): # Learning rate update function\n if epoch < lr_ramp_ep: lr = (lr_max - lr_start) / lr_ramp_ep * epoch + lr_start\n elif epoch < lr_ramp_ep + lr_sus_ep: lr = lr_max\n elif mode == 'exp': lr = (lr_max - lr_min) * lr_decay**(epoch - lr_ramp_ep - lr_sus_ep) + lr_min\n elif mode == 'step': lr = lr_max * lr_decay**((epoch - lr_ramp_ep - lr_sus_ep) // 2)\n elif mode == 'cos':\n decay_total_epochs, decay_epoch_index = epochs - lr_ramp_ep - lr_sus_ep + 3, epoch - lr_ramp_ep - lr_sus_ep\n phase = math.pi * decay_epoch_index / decay_total_epochs\n lr = (lr_max - lr_min) * 0.5 * (1 + math.cos(phase)) + lr_min\n return lr\n\n if plot: # Plot lr curve if plot is True\n plt.figure(figsize=(10, 5))\n plt.plot(np.arange(epochs), [lrfn(epoch) for epoch in np.arange(epochs)], marker='o')\n plt.xlabel('epoch'); plt.ylabel('lr')\n plt.title('LR Scheduler')\n plt.show()\n\n return keras.callbacks.LearningRateScheduler(lrfn, verbose=False) # Create lr callback","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.510014,"end_time":"2022-03-08T03:20:45.290695","exception":false,"start_time":"2022-03-08T03:20:44.780681","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-05T07:09:05.304041Z","iopub.execute_input":"2024-06-05T07:09:05.304433Z","iopub.status.idle":"2024-06-05T07:09:05.315062Z","shell.execute_reply.started":"2024-06-05T07:09:05.304399Z","shell.execute_reply":"2024-06-05T07:09:05.314053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr_cb = get_lr_callback(CFG.batch_size, plot=True)","metadata":{"execution":{"iopub.status.busy":"2024-06-05T07:09:11.029370Z","iopub.execute_input":"2024-06-05T07:09:11.030202Z","iopub.status.idle":"2024-06-05T07:09:11.238070Z","shell.execute_reply.started":"2024-06-05T07:09:11.030168Z","shell.execute_reply":"2024-06-05T07:09:11.237136Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Checkpoint 💾","metadata":{}},{"cell_type":"code","source":"ckpt_cb = keras.callbacks.ModelCheckpoint(\"VGG16best_model.weights.h5\",\n monitor='val_auc',\n save_best_only=True,\n save_weights_only=True,\n mode='max')","metadata":{"execution":{"iopub.status.busy":"2024-06-05T07:09:21.181195Z","iopub.execute_input":"2024-06-05T07:09:21.182143Z","iopub.status.idle":"2024-06-05T07:09:21.186852Z","shell.execute_reply.started":"2024-06-05T07:09:21.182106Z","shell.execute_reply":"2024-06-05T07:09:21.185769Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training 🚄","metadata":{}},{"cell_type":"code","source":"history = model.fit(\n train_ds, \n validation_data=valid_ds, \n epochs=CFG.epochs,\n callbacks=[lr_cb, ckpt_cb], \n verbose=1\n)","metadata":{"execution":{"iopub.status.busy":"2024-06-05T07:09:26.498643Z","iopub.execute_input":"2024-06-05T07:09:26.499035Z","iopub.status.idle":"2024-06-05T08:40:40.533666Z","shell.execute_reply.started":"2024-06-05T07:09:26.499004Z","shell.execute_reply":"2024-06-05T08:40:40.532649Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.save(\"/kaggle/working/vgg16my_model.keras\") ","metadata":{"execution":{"iopub.status.busy":"2024-06-05T08:48:12.781254Z","iopub.execute_input":"2024-06-05T08:48:12.781624Z","iopub.status.idle":"2024-06-05T08:48:13.552474Z","shell.execute_reply.started":"2024-06-05T08:48:12.781593Z","shell.execute_reply":"2024-06-05T08:48:13.551676Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Result Summary","metadata":{}},{"cell_type":"code","source":"best_epoch = np.argmax(history.history[\"val_auc\"])\nbest_score = history.history[\"val_auc\"][best_epoch]\nprint('>>> Best AUC: ', best_score)\nprint('>>> Best Epoch: ', best_epoch+1)","metadata":{"execution":{"iopub.status.busy":"2024-06-05T08:48:15.483406Z","iopub.execute_input":"2024-06-05T08:48:15.484260Z","iopub.status.idle":"2024-06-05T08:48:15.489877Z","shell.execute_reply.started":"2024-06-05T08:48:15.484217Z","shell.execute_reply":"2024-06-05T08:48:15.488886Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Reference ✍️\n* [BirdCLEF23: Pretraining is All you Need [Train]](https://www.kaggle.com/code/awsaf49/birdclef23-pretraining-is-all-you-need-train) by @awsaf49\n* [RANZCR: EfficientNet TPU Training](https://www.kaggle.com/code/xhlulu/ranzcr-efficientnet-tpu-training) by @xhlulu\n* [Triple Stratified KFold with TFRecords](https://www.kaggle.com/code/cdeotte/triple-stratified-kfold-with-tfrecords) by @cdeotte","metadata":{}}]}
![\"Keras](\"https://keras.io/img/logo-small.png\"){kind=logo}
![\"BirdCLEF](\"https://i.ibb.co/47F4P9R/birdclef2024.png\")
\n