{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":70203,"databundleVersionId":8068726,"sourceType":"competition"},{"sourceId":172963793,"sourceType":"kernelVersion"},{"sourceId":6125,"sourceType":"modelInstanceVersion","modelInstanceId":4596},{"sourceId":6127,"sourceType":"modelInstanceVersion","modelInstanceId":4598}],"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"
\"Keras
\nThis starter notebook is provided by the Keras team.
","metadata":{}},{"cell_type":"markdown","source":"## This notebook accidentally used the default weights of the default infer notebook as opposed to the weights trained specifically for it.\n\n## This notebook not only had better results with the wrong weights, but also the best weights in general.","metadata":{}},{"cell_type":"markdown","source":"# BirdCLEF 2024 with [KerasCV](https://github.com/keras-team/keras-cv) and [Keras](https://github.com/keras-team/keras)\n\n> The objective of this competition is to identify under-studied Indian bird species by their calls.\n\n
\n \n
\n\nThis notebook guides you through the process of inferring a Deep Learning model to recognize bird species by their songs (audio data). As the inference requires running only on the `CPU`, we had to create a separate notebooks for training and inference. You can find the [training notebook here](https://www.kaggle.com/code/awsaf49/birdclef24-kerascv-starter-train). Just as a recap of the training notebook, it uses the EfficientNetV2 backbone from KerasCV on the competition dataset. That notebook also demonstrates how to convert audio data to mel-spectrograms using Keras.\n\nFun fact: Both the training and inference notebooks are backend-agnostic, supporting TensorFlow, PyTorch, and JAX. Utilizing KerasCV and Keras allows us to choose our preferred backend. Explore more details on [Keras](https://keras.io/keras_core/announcement/).\n\nIn this notebook, you will learn:\n\n- Designing a data pipeline for audio data, including audio-to-spectrogram conversion.\n- Loading the data efficiently using [`tf.data`](https://www.tensorflow.org/guide/data).\n- Creating the model using KerasCV presets.\n- Inferring the trained model.\n\n**Note**: For a more in-depth understanding of KerasCV, refer to the [KerasCV guides](https://keras.io/guides/keras_cv/).","metadata":{}},{"cell_type":"markdown","source":"# Import Libraries 📚","metadata":{"papermill":{"duration":0.065343,"end_time":"2022-03-08T03:18:11.885586","exception":false,"start_time":"2022-03-08T03:18:11.820243","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import os\nos.environ[\"KERAS_BACKEND\"] = \"tensorflow\" # \"jax\" or \"tensorflow\" or \"torch\" \n\nimport keras_cv\nimport keras\nimport keras.backend as K\nimport tensorflow as tf\nimport tensorflow_io as tfio\n\nimport numpy as np \nimport pandas as pd\n\nfrom glob import glob\nfrom tqdm import tqdm\n\nimport librosa\nimport IPython.display as ipd\nimport librosa.display as lid\n\nimport matplotlib.pyplot as plt\nimport matplotlib as mpl\n\ncmap = mpl.cm.get_cmap('coolwarm')","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","papermill":{"duration":2.632068,"end_time":"2022-03-08T03:18:14.585094","exception":false,"start_time":"2022-03-08T03:18:11.953026","status":"completed"},"tags":[],"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-06-14T12:38:19.729206Z","iopub.execute_input":"2024-06-14T12:38:19.730045Z","iopub.status.idle":"2024-06-14T12:38:45.509315Z","shell.execute_reply.started":"2024-06-14T12:38:19.730008Z","shell.execute_reply":"2024-06-14T12:38:45.507906Z"},"trusted":true},"execution_count":1,"outputs":[{"name":"stderr","text":"2024-06-14 12:38:26.452968: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n2024-06-14 12:38:26.453101: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n2024-06-14 12:38:26.656373: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n/tmp/ipykernel_33/1585041062.py:23: MatplotlibDeprecationWarning: The get_cmap function was deprecated in Matplotlib 3.7 and will be removed two minor releases later. Use ``matplotlib.colormaps[name]`` or ``matplotlib.colormaps.get_cmap(obj)`` instead.\n cmap = mpl.cm.get_cmap('coolwarm')\n","output_type":"stream"}]},{"cell_type":"markdown","source":"# Configuration ⚙️","metadata":{"papermill":{"duration":0.066353,"end_time":"2022-03-08T03:18:18.099835","exception":false,"start_time":"2022-03-08T03:18:18.033482","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class CFG:\n seed = 42\n \n # Input image size and batch size\n img_size = [128, 384]\n \n # Audio duration, sample rate, and length\n #duration = 15 # second\n #sample_rate = 32000\n duration = 10 # second\n sample_rate = 64000\n audio_len = duration*sample_rate\n \n # STFT parameters\n nfft = 2028\n window = 2048\n hop_length = audio_len // (img_size[1] - 1)\n fmin = 20\n fmax = 16000\n \n # Number of epochs, model name\n preset = 'efficientnetv2_b2_imagenet'\n\n # Class Labels for BirdCLEF 24\n class_names = sorted(os.listdir('/kaggle/input/birdclef-2024/train_audio/'))\n num_classes = len(class_names)\n class_labels = list(range(num_classes))\n label2name = dict(zip(class_labels, class_names))\n name2label = {v:k for k,v in label2name.items()}","metadata":{"papermill":{"duration":0.156464,"end_time":"2022-03-08T03:18:18.322809","exception":false,"start_time":"2022-03-08T03:18:18.166345","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T12:38:45.512259Z","iopub.execute_input":"2024-06-14T12:38:45.513173Z","iopub.status.idle":"2024-06-14T12:38:45.560208Z","shell.execute_reply.started":"2024-06-14T12:38:45.513111Z","shell.execute_reply":"2024-06-14T12:38:45.558781Z"},"trusted":true},"execution_count":2,"outputs":[]},{"cell_type":"markdown","source":"# Reproducibility ♻️\nSets value for random seed to produce similar result in each run.","metadata":{"papermill":{"duration":0.070351,"end_time":"2022-03-08T03:18:18.46058","exception":false,"start_time":"2022-03-08T03:18:18.390229","status":"completed"},"tags":[]}},{"cell_type":"code","source":"tf.keras.utils.set_random_seed(CFG.seed)","metadata":{"papermill":{"duration":0.153451,"end_time":"2022-03-08T03:18:18.685056","exception":false,"start_time":"2022-03-08T03:18:18.531605","status":"completed"},"tags":[],"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-06-14T12:38:45.568655Z","iopub.execute_input":"2024-06-14T12:38:45.569187Z","iopub.status.idle":"2024-06-14T12:38:45.576185Z","shell.execute_reply.started":"2024-06-14T12:38:45.569138Z","shell.execute_reply":"2024-06-14T12:38:45.574242Z"},"trusted":true},"execution_count":3,"outputs":[]},{"cell_type":"markdown","source":"# Dataset Path 📁","metadata":{}},{"cell_type":"code","source":"BASE_PATH = '/kaggle/input/birdclef-2024'","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-06-14T12:38:45.578076Z","iopub.execute_input":"2024-06-14T12:38:45.578549Z","iopub.status.idle":"2024-06-14T12:38:45.609321Z","shell.execute_reply.started":"2024-06-14T12:38:45.578509Z","shell.execute_reply":"2024-06-14T12:38:45.607901Z"},"trusted":true},"execution_count":4,"outputs":[]},{"cell_type":"markdown","source":"# Test Data 📖","metadata":{"papermill":{"duration":0.067107,"end_time":"2022-03-08T03:18:26.962626","exception":false,"start_time":"2022-03-08T03:18:26.895519","status":"completed"},"tags":[]}},{"cell_type":"code","source":"test_paths = glob(f'{BASE_PATH}/test_soundscapes/*ogg')\n# During commit use `unlabeled` data as there is no `test` data.\n# During submission `test` data will automatically be populated.\nif len(test_paths)==0:\n test_paths = glob(f'{BASE_PATH}/unlabeled_soundscapes/*ogg')[:10]\ntest_df = pd.DataFrame(test_paths, columns=['filepath'])\ntest_df.head()","metadata":{"papermill":{"duration":0.241649,"end_time":"2022-03-08T03:18:27.408813","exception":false,"start_time":"2022-03-08T03:18:27.167164","status":"completed"},"tags":[],"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-06-14T12:38:45.611386Z","iopub.execute_input":"2024-06-14T12:38:45.611899Z","iopub.status.idle":"2024-06-14T12:38:46.010563Z","shell.execute_reply.started":"2024-06-14T12:38:45.611855Z","shell.execute_reply":"2024-06-14T12:38:46.009308Z"},"trusted":true},"execution_count":5,"outputs":[{"execution_count":5,"output_type":"execute_result","data":{"text/plain":" filepath\n0 /kaggle/input/birdclef-2024/unlabeled_soundsca...\n1 /kaggle/input/birdclef-2024/unlabeled_soundsca...\n2 /kaggle/input/birdclef-2024/unlabeled_soundsca...\n3 /kaggle/input/birdclef-2024/unlabeled_soundsca...\n4 /kaggle/input/birdclef-2024/unlabeled_soundsca...","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
filepath
0/kaggle/input/birdclef-2024/unlabeled_soundsca...
1/kaggle/input/birdclef-2024/unlabeled_soundsca...
2/kaggle/input/birdclef-2024/unlabeled_soundsca...
3/kaggle/input/birdclef-2024/unlabeled_soundsca...
4/kaggle/input/birdclef-2024/unlabeled_soundsca...
\n
"},"metadata":{}}]},{"cell_type":"markdown","source":"# Modeling 🤖\n\nNote that our model was trained on `10 second` duration audio files, but we will infer on `5-second` audio files (as per competition rules). To facilitate this, we have set the model input shape to `(None, None, 3)`, which will allow us to have variable-length input during training and inference.","metadata":{"papermill":{"duration":0.182769,"end_time":"2022-03-08T03:20:04.861966","exception":false,"start_time":"2022-03-08T03:20:04.679197","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Create an input layer for the model\ninp = keras.layers.Input(shape=(None, None, 3))\n# Pretrained backbone\nbackbone = keras_cv.models.EfficientNetV2Backbone.from_preset(\n CFG.preset,\n)\nout = keras_cv.models.ImageClassifier(\n backbone=backbone,\n num_classes=CFG.num_classes,\n name=\"classifier\"\n)(inp)\n# Build model\nmodel = keras.models.Model(inputs=inp, outputs=out)\n# Load weights of trained model\n#model.load_weights(\"/kaggle/input/birdclef24-kerascv-starter-train/best_model.weights.h5\")\nmodel.load_weights(\"/kaggle/input/birdclef24-kerascv-starter-train/best_model.weights.h5\")","metadata":{"papermill":{"duration":1.239321,"end_time":"2022-03-08T03:20:06.281118","exception":false,"start_time":"2022-03-08T03:20:05.041797","status":"completed"},"tags":[],"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-06-14T12:38:46.011891Z","iopub.execute_input":"2024-06-14T12:38:46.012225Z","iopub.status.idle":"2024-06-14T12:38:57.557127Z","shell.execute_reply.started":"2024-06-14T12:38:46.012197Z","shell.execute_reply":"2024-06-14T12:38:57.556001Z"},"trusted":true},"execution_count":6,"outputs":[{"name":"stderr","text":"Attaching 'config.json' from model 'keras/efficientnetv2/keras/efficientnetv2_b2_imagenet/2' to your Kaggle notebook...\nAttaching 'config.json' from model 'keras/efficientnetv2/keras/efficientnetv2_b2_imagenet/2' to your Kaggle notebook...\nAttaching 'model.weights.h5' from model 'keras/efficientnetv2/keras/efficientnetv2_b2_imagenet/2' to your Kaggle notebook...\n","output_type":"stream"}]},{"cell_type":"markdown","source":"# Data Loader 🍚\n\nThe following code will decode the raw audio from `.ogg` file and also decode the spectrogram from the `audio` file. Additionally, we will apply Z-Score standardization and Min-Max normalization to ensure consistent inputs to the model.","metadata":{}},{"cell_type":"code","source":"# Decodes Audio\ndef build_decoder(with_labels=True, dim=1024):\n def get_audio(filepath):\n file_bytes = tf.io.read_file(filepath)\n audio = tfio.audio.decode_vorbis(file_bytes) # decode .ogg file\n audio = tf.cast(audio, tf.float32)\n if tf.shape(audio)[1]>1: # stereo -> mono\n audio = audio[...,0:1]\n audio = tf.squeeze(audio, axis=-1)\n return audio\n \n def create_frames(audio, duration=5, sr=32000):\n frame_size = int(duration * sr)\n audio = tf.pad(audio[..., None], [[0, tf.shape(audio)[0] % frame_size], [0, 0]]) # pad the end\n audio = tf.squeeze(audio) # remove extra dimension added for padding\n frames = tf.reshape(audio, [-1, frame_size]) # shape: [num_frames, frame_size]\n return frames\n \n def apply_preproc(spec):\n # Standardize\n mean = tf.math.reduce_mean(spec)\n std = tf.math.reduce_std(spec)\n spec = tf.where(tf.math.equal(std, 0), spec - mean, (spec - mean) / std)\n\n # Normalize using Min-Max\n min_val = tf.math.reduce_min(spec)\n max_val = tf.math.reduce_max(spec)\n spec = tf.where(tf.math.equal(max_val - min_val, 0), spec - min_val,\n (spec - min_val) / (max_val - min_val))\n return spec\n\n def decode(path):\n # Load audio file\n audio = get_audio(path)\n # Split audio file into frames with each having 5 seecond duration\n audio = create_frames(audio)\n # Convert audio to spectrogram\n spec = keras.layers.MelSpectrogram(num_mel_bins=CFG.img_size[0],\n fft_length=CFG.nfft, \n sequence_stride=CFG.hop_length, \n sampling_rate=CFG.sample_rate)(audio)\n # Apply normalization and standardization\n spec = apply_preproc(spec)\n # Covnert spectrogram to 3 channel image (for imagenet)\n spec = tf.tile(spec[..., None], [1, 1, 1, 3])\n return spec\n \n return decode","metadata":{"execution":{"iopub.status.busy":"2024-06-14T12:38:57.559417Z","iopub.execute_input":"2024-06-14T12:38:57.559937Z","iopub.status.idle":"2024-06-14T12:38:57.577817Z","shell.execute_reply.started":"2024-06-14T12:38:57.559892Z","shell.execute_reply":"2024-06-14T12:38:57.576233Z"},"trusted":true},"execution_count":7,"outputs":[]},{"cell_type":"code","source":"# Build data loader\ndef build_dataset(paths, batch_size=1, decode_fn=None, cache=False):\n if decode_fn is None:\n decode_fn = build_decoder(dim=CFG.audio_len) # decoder\n AUTO = tf.data.experimental.AUTOTUNE\n slices = (paths,)\n ds = tf.data.Dataset.from_tensor_slices(slices)\n ds = ds.map(decode_fn, num_parallel_calls=AUTO) # decode audio to spectrograms then create frames\n ds = ds.cache() if cache else ds # cache files\n ds = ds.batch(batch_size, drop_remainder=False) # create batches\n ds = ds.prefetch(AUTO)\n return ds","metadata":{"execution":{"iopub.status.busy":"2024-06-14T12:38:57.579894Z","iopub.execute_input":"2024-06-14T12:38:57.580306Z","iopub.status.idle":"2024-06-14T12:38:57.59965Z","shell.execute_reply.started":"2024-06-14T12:38:57.580274Z","shell.execute_reply":"2024-06-14T12:38:57.598225Z"},"trusted":true},"execution_count":8,"outputs":[]},{"cell_type":"markdown","source":"# Inference 🏃","metadata":{}},{"cell_type":"code","source":"# Initialize empty list to store ids\nids = []\n\n# Initialize empty array to store predictions\npreds = np.empty(shape=(0, CFG.num_classes), dtype='float32')\n\n# Build test dataset\ntest_paths = test_df.filepath.tolist()\ntest_ds = build_dataset(paths=test_paths, batch_size=1)\n\n# Iterate over each audio file in the test dataset\nfor idx, specs in enumerate(tqdm(iter(test_ds), desc='test ', total=len(test_df))):\n # Extract the filename without the extension\n filename = test_paths[idx].split('/')[-1].replace('.ogg','')\n \n # Convert to backend-specific tensor while excluding extra dimension\n specs = keras.ops.convert_to_tensor(specs[0])\n \n # Predict bird species for all frames in a recording using all trained models\n frame_preds = model.predict(specs, verbose=0)\n \n # Create a ID for each frame in a recording using the filename and frame number\n frame_ids = [f'{filename}_{(frame_id+1)*5}' for frame_id in range(len(frame_preds))]\n \n # Concatenate the ids\n ids += frame_ids\n # Concatenate the predictions\n preds = np.concatenate([preds, frame_preds], axis=0)","metadata":{"execution":{"iopub.status.busy":"2024-06-14T12:38:57.60294Z","iopub.execute_input":"2024-06-14T12:38:57.603415Z","iopub.status.idle":"2024-06-14T12:39:19.894507Z","shell.execute_reply.started":"2024-06-14T12:38:57.603378Z","shell.execute_reply":"2024-06-14T12:39:19.893295Z"},"trusted":true},"execution_count":9,"outputs":[{"name":"stderr","text":"test : 100%|██████████| 10/10 [00:20<00:00, 2.03s/it]\n","output_type":"stream"}]},{"cell_type":"markdown","source":"# Submission ✉️","metadata":{}},{"cell_type":"code","source":"# Submit prediction\npred_df = pd.DataFrame(ids, columns=['row_id'])\npred_df.loc[:, CFG.class_names] = preds\npred_df.to_csv('submission.csv',index=False)\npred_df.head()","metadata":{"execution":{"iopub.status.busy":"2024-06-14T12:39:19.896205Z","iopub.execute_input":"2024-06-14T12:39:19.896599Z","iopub.status.idle":"2024-06-14T12:39:20.21223Z","shell.execute_reply.started":"2024-06-14T12:39:19.896568Z","shell.execute_reply":"2024-06-14T12:39:20.211015Z"},"trusted":true},"execution_count":10,"outputs":[{"execution_count":10,"output_type":"execute_result","data":{"text/plain":" row_id asbfly ashdro1 ashpri1 ashwoo2 asikoe2 asiope1 \\\n0 1872382287_5 0.004806 0.004003 0.000612 0.000740 0.002018 0.000650 \n1 1872382287_10 0.003871 0.003612 0.000711 0.001102 0.001471 0.000407 \n2 1872382287_15 0.001201 0.004474 0.000428 0.000539 0.005514 0.000241 \n3 1872382287_20 0.003099 0.003052 0.000397 0.000561 0.005135 0.000968 \n4 1872382287_25 0.002020 0.003128 0.000510 0.000541 0.004951 0.000793 \n\n aspfly1 aspswi1 barfly1 ... whbwoo2 whcbar1 whiter2 whrmun \\\n0 0.002482 0.000823 0.000414 ... 0.002422 0.000638 0.003146 0.004793 \n1 0.004750 0.000854 0.000338 ... 0.005328 0.000652 0.002209 0.003805 \n2 0.002553 0.000553 0.000222 ... 0.009863 0.000514 0.001392 0.002053 \n3 0.001993 0.000713 0.000332 ... 0.017493 0.000838 0.001116 0.001169 \n4 0.002102 0.001621 0.000610 ... 0.008820 0.001368 0.003230 0.002846 \n\n whtkin2 woosan wynlau1 yebbab1 yebbul3 zitcis1 \n0 0.008208 0.005590 0.000133 0.000415 0.000549 0.002978 \n1 0.002646 0.002518 0.000242 0.000356 0.000723 0.003035 \n2 0.003868 0.002291 0.000182 0.000201 0.000742 0.001123 \n3 0.003730 0.001328 0.000323 0.000403 0.000925 0.000980 \n4 0.004489 0.005407 0.000348 0.000403 0.000943 0.001174 \n\n[5 rows x 183 columns]","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
row_idasbflyashdro1ashpri1ashwoo2asikoe2asiope1aspfly1aspswi1barfly1...whbwoo2whcbar1whiter2whrmunwhtkin2woosanwynlau1yebbab1yebbul3zitcis1
01872382287_50.0048060.0040030.0006120.0007400.0020180.0006500.0024820.0008230.000414...0.0024220.0006380.0031460.0047930.0082080.0055900.0001330.0004150.0005490.002978
11872382287_100.0038710.0036120.0007110.0011020.0014710.0004070.0047500.0008540.000338...0.0053280.0006520.0022090.0038050.0026460.0025180.0002420.0003560.0007230.003035
21872382287_150.0012010.0044740.0004280.0005390.0055140.0002410.0025530.0005530.000222...0.0098630.0005140.0013920.0020530.0038680.0022910.0001820.0002010.0007420.001123
31872382287_200.0030990.0030520.0003970.0005610.0051350.0009680.0019930.0007130.000332...0.0174930.0008380.0011160.0011690.0037300.0013280.0003230.0004030.0009250.000980
41872382287_250.0020200.0031280.0005100.0005410.0049510.0007930.0021020.0016210.000610...0.0088200.0013680.0032300.0028460.0044890.0054070.0003480.0004030.0009430.001174
\n

5 rows × 183 columns

\n
"},"metadata":{}}]},{"cell_type":"markdown","source":"# Reference ✍️\n* [Fake Speech Detection: Conformer [TF]](https://www.kaggle.com/code/awsaf49/fake-speech-detection-conformer-tf/) by @awsaf49\n* [RANZCR: EfficientNet TPU Training](https://www.kaggle.com/code/xhlulu/ranzcr-efficientnet-tpu-training) by @xhlulu\n* [Triple Stratified KFold with TFRecords](https://www.kaggle.com/code/cdeotte/triple-stratified-kfold-with-tfrecords) by @cdeotte","metadata":{}}]}