{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Bird Species Classification using TensorFlow Hub Model\n\nThis code performs bird species classification in audio recordings using a TensorFlow Hub model for bird vocalization classification. It uses the BirdCLEF 2023 dataset and employs various functions to load audio files, load the model and labels, frame audio data, resample waveforms, and make predictions.\n\n## Functionality\n\n- **Load Audio Files**: The `load_audio_files()` function loads and plays audio files from the specified directory.\n- **Load Model**: The `load_model()` function loads the TensorFlow Hub model and its corresponding labels.\n- **Class Names from CSV**: The `class_names_from_csv()` function retrieves the list of class names corresponding to the score vector.\n- **Load Train Metadata**: The `load_train_metadata()` function loads the training metadata, specifically the unique primary labels.\n- **Frame Audio**: The `frame_audio()` function frames audio data into fixed-size windows for inference.\n- **Ensure Sample Rate**: The `ensure_sample_rate()` function ensures that the waveform has the desired sample rate.\n- **Predict for Sample**: The `predict_for_sample()` function predicts the class probabilities for each window of the audio sample.\n- **Main Function**: The `main()` function orchestrates the entire process, including loading audio files, making predictions, and generating the submission file.\n\n## TensorFlow Hub Model - Bird Vocalization Classifier\n\nThe code utilizes a pre-trained TensorFlow Hub model for bird vocalization classification. The model path and label CSV file are specified to load the model and labels.\n\nImportant Parameters:\n- `model_path`: The path to the TensorFlow Hub model for bird vocalization classification.\n- `labels_path`: The path to the label CSV file corresponding to the model.\n\n       +-------------------------+\n       | TensorFlow Hub Model   |\n       | (Bird Vocalization     |\n       | Classification Model)  |\n       +----------^--------------+\n                  |\n                  |\n       +----------v--------------+\n       |    Label CSV File       |\n       +-------------------------+\n","metadata":{}},{"cell_type":"markdown","source":"# Import libraries\n","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport librosa\nimport glob\nimport csv\nimport tensorflow as tf\nimport tensorflow_hub as hub\nfrom scipy.signal import resample\nfrom IPython.display import Audio","metadata":{"execution":{"iopub.status.busy":"2023-06-26T08:32:44.593560Z","iopub.execute_input":"2023-06-26T08:32:44.594016Z","iopub.status.idle":"2023-06-26T08:32:55.251862Z","shell.execute_reply.started":"2023-06-26T08:32:44.593978Z","shell.execute_reply":"2023-06-26T08:32:55.250837Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Function to load and play audio files\n","metadata":{}},{"cell_type":"code","source":"def load_audio_files():\n    # Loading and playing audio files\n    path = \"/kaggle/input/birdclef-2023/train_audio/\"\n    bird_species = [\"abethr1/XC128013.ogg\", \"abhori1/XC127317.ogg\", \"edcsun3/XC470591.ogg\"]\n    \n    for bird in bird_species:\n        audio, sr = librosa.load(path + bird)\n        print(Audio(data=audio, rate=sr))","metadata":{"execution":{"iopub.status.busy":"2023-06-26T08:32:55.254903Z","iopub.execute_input":"2023-06-26T08:32:55.256398Z","iopub.status.idle":"2023-06-26T08:32:55.264421Z","shell.execute_reply.started":"2023-06-26T08:32:55.256323Z","shell.execute_reply":"2023-06-26T08:32:55.262637Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Function to load the model and labels\n","metadata":{}},{"cell_type":"code","source":"def load_model():\n    # Load the model and labels\n    model_path = '/kaggle/input/bird-vocalization-classifier/tensorflow2/bird-vocalization-classifier/2'\n    model = hub.load(model_path)\n    labels_path = hub.resolve(model_path) + \"/assets/label.csv\"\n    \n    return model, labels_path","metadata":{"execution":{"iopub.status.busy":"2023-06-26T08:32:55.266460Z","iopub.execute_input":"2023-06-26T08:32:55.267537Z","iopub.status.idle":"2023-06-26T08:32:55.282545Z","shell.execute_reply.started":"2023-06-26T08:32:55.267502Z","shell.execute_reply":"2023-06-26T08:32:55.281353Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Function to retrieve class names from a CSV file\n","metadata":{}},{"cell_type":"code","source":"def class_names_from_csv(class_map_csv_text):\n    # Returns list of class names corresponding to score vector\n    with open(class_map_csv_text) as csv_file:\n        csv_reader = csv.reader(csv_file, delimiter=',')\n        class_names = [mid for mid, desc in csv_reader][1:]\n    return class_names","metadata":{"execution":{"iopub.status.busy":"2023-06-26T08:32:55.285702Z","iopub.execute_input":"2023-06-26T08:32:55.286173Z","iopub.status.idle":"2023-06-26T08:32:55.298118Z","shell.execute_reply.started":"2023-06-26T08:32:55.286134Z","shell.execute_reply":"2023-06-26T08:32:55.296506Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Function to load the competition classes from the train metadata\n","metadata":{}},{"cell_type":"code","source":"def load_train_metadata():\n    # Load train metadata\n    train_metadata = pd.read_csv(\"/kaggle/input/birdclef-2023/train_metadata.csv\")\n    competition_classes = sorted(train_metadata.primary_label.unique())\n    return competition_classes","metadata":{"execution":{"iopub.status.busy":"2023-06-26T08:32:55.300126Z","iopub.execute_input":"2023-06-26T08:32:55.300563Z","iopub.status.idle":"2023-06-26T08:32:55.311836Z","shell.execute_reply.started":"2023-06-26T08:32:55.300521Z","shell.execute_reply":"2023-06-26T08:32:55.310680Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Function to frame audio into fixed-size windows\n","metadata":{}},{"cell_type":"code","source":"def frame_audio(audio_array: np.ndarray, window_size_s: float = 5.0, hop_size_s: float = 5.0, sample_rate=32000) -> np.ndarray:\n    # Framing audio for inference\n    if window_size_s is None or window_size_s < 0:\n        return audio_array[np.newaxis, :]\n    frame_length = int(window_size_s * sample_rate)\n    hop_length = int(hop_size_s * sample_rate)\n    framed_audio = tf.signal.frame(audio_array, frame_length, hop_length, pad_end=True)\n    return framed_audio","metadata":{"execution":{"iopub.status.busy":"2023-06-26T08:32:55.313539Z","iopub.execute_input":"2023-06-26T08:32:55.313973Z","iopub.status.idle":"2023-06-26T08:32:55.325435Z","shell.execute_reply.started":"2023-06-26T08:32:55.313934Z","shell.execute_reply":"2023-06-26T08:32:55.324205Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Function to ensure the desired sample rate for audio\n","metadata":{}},{"cell_type":"code","source":"def ensure_sample_rate(waveform, original_sample_rate, desired_sample_rate=32000):\n    if original_sample_rate != desired_sample_rate:\n        num_samples = len(waveform)\n        original_duration = num_samples / original_sample_rate\n        time_old = np.linspace(0, original_duration, num_samples)\n        time_new = np.linspace(0, original_duration, int(num_samples * desired_sample_rate / original_sample_rate))\n        waveform = resample(waveform, int(num_samples * desired_sample_rate / original_sample_rate))\n        sample_rate = desired_sample_rate\n    return sample_rate, waveform","metadata":{"execution":{"iopub.status.busy":"2023-06-26T08:32:55.326968Z","iopub.execute_input":"2023-06-26T08:32:55.327335Z","iopub.status.idle":"2023-06-26T08:32:55.339931Z","shell.execute_reply.started":"2023-06-26T08:32:55.327305Z","shell.execute_reply":"2023-06-26T08:32:55.338780Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Function to predict class probabilities for a sample\n","metadata":{}},{"cell_type":"code","source":"def predict_for_sample(filename, sample_submission, frame_limit_secs=None):\n    # Get the file ID from the filename\n    file_id = filename.split(\".ogg\")[0].split(\"/\")[-1]\n\n    # Load the audio data and ensure the correct sample rate\n    audio, sample_rate = librosa.load(filename)\n    sample_rate, wav_data = ensure_sample_rate(audio, sample_rate)\n\n    # Frame the audio data into fixed-size windows\n    fixed_tm = frame_audio(wav_data)\n\n    # Load the model and labels\n    model, labels_path = load_model()\n    classes = class_names_from_csv(labels_path)\n    competition_classes = load_train_metadata()\n    competition_class_map = [classes.index(c) if c in classes else 0 for c in competition_classes]\n\n    # Predict the class probabilities for each window\n    frame = 5\n    all_logits, all_embeddings = model.infer_tf(fixed_tm[:1])\n    for window in fixed_tm[1:]:\n        if frame_limit_secs and frame > frame_limit_secs:\n            continue\n\n        # Infer the logits and embeddings for the current window\n        logits, embeddings = model.infer_tf(window[np.newaxis, :])\n\n        # Append the logits to the array of all logits\n        all_logits = np.concatenate([all_logits, logits], axis=0)\n\n        # Increment the frame counter\n        frame += 5\n\n    # Convert the logits to probabilities and update the sample submission\n    frame = 5\n    for frame_logits in all_logits:\n        probabilities = tf.nn.softmax(frame_logits).numpy()\n\n        # set the appropriate row in the sample submission\n        sample_submission.loc[sample_submission.row_id == file_id + \"_\" + str(frame), competition_classes] = probabilities[competition_class_map]\n\n        # Increment the frame counter\n        frame += 5","metadata":{"execution":{"iopub.status.busy":"2023-06-26T08:32:55.341576Z","iopub.execute_input":"2023-06-26T08:32:55.341928Z","iopub.status.idle":"2023-06-26T08:32:55.360709Z","shell.execute_reply.started":"2023-06-26T08:32:55.341898Z","shell.execute_reply":"2023-06-26T08:32:55.359580Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Main function\n","metadata":{}},{"cell_type":"code","source":"def main():\n    # Load the audio files\n    load_audio_files()\n\n    # Load the test samples\n    test_samples = glob.glob(\"/kaggle/input/birdclef-2023/test_soundscapes/*.ogg\")\n\n    # Load the sample submission file\n    sample_sub = pd.read_csv(\"/kaggle/input/birdclef-2023/sample_submission.csv\")\n\n    # Load competition classes\n    competition_classes = load_train_metadata()\n\n    # Set the desired frame limit in seconds\n    frame_limit_secs = 15 if sample_sub.shape[0] == 3 else None\n\n    # Iterate over test samples and make predictions\n    for sample_filename in test_samples:\n        predict_for_sample(sample_filename, sample_sub, frame_limit_secs=frame_limit_secs)\n\n    # Save the submission file\n    sample_sub.to_csv(\"submission.csv\", index=False)\n","metadata":{"execution":{"iopub.status.busy":"2023-06-26T08:32:55.362305Z","iopub.execute_input":"2023-06-26T08:32:55.362796Z","iopub.status.idle":"2023-06-26T08:32:55.380440Z","shell.execute_reply.started":"2023-06-26T08:32:55.362767Z","shell.execute_reply":"2023-06-26T08:32:55.379203Z"},"trusted":true},"execution_count":null,"outputs":[]}]}