{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":44224,"databundleVersionId":5188730,"sourceType":"competition"}],"dockerImageVersionId":30587,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Bird Call Classification\n## Why this project?\n### Environmental Justification\nBirds are at the top of food chain, hence monitoring them alone gives a decent picture of what is happening in the lower levels of the food chain. Birds are wonderful indicators of the habitat's quality. Now, it is easier to hear birds than see them. Hence, with proper bird sound detection and classification, researchers can bring up conclusions about an area's quality of life based on a chaging bird population.\n\n### Deep Learning Justification\n1. The task at hand is a perfect example of why Machine Learning is popular today. There is no other field or branch of science which can perform this task.\n2. The data used is a wild dataset, deep learning works great for such dataset. The term \"wild dataset\" in the context of deep learning generally refers to datasets that are collected from real-world, diverse, and uncontrolled environments. These datasets are often contrasted with \"controlled\" datasets that are carefully curated or designed under specific conditions.","metadata":{}},{"cell_type":"markdown","source":"# Package Installation","metadata":{}},{"cell_type":"code","source":"!pip install noisereduce","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:16:22.021986Z","iopub.execute_input":"2023-11-24T13:16:22.022703Z","iopub.status.idle":"2023-11-24T13:16:34.651115Z","shell.execute_reply.started":"2023-11-24T13:16:22.022671Z","shell.execute_reply":"2023-11-24T13:16:34.650131Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---","metadata":{}},{"cell_type":"markdown","source":"# Libraries Required\nNote: - The libraries are imported sequential, when they are required. ","metadata":{}},{"cell_type":"code","source":"import warnings # we don't want any warnings to show up \nwarnings.filterwarnings('ignore')\nimport numpy as np # linear algebra\nimport pandas as pd # data processing\nimport plotly.express as px # plotting graph\nimport matplotlib.pyplot as plt # display images\n%matplotlib inline\nimport librosa # for audio loading\nfrom IPython.display import Audio # for playback\nfrom PIL import Image # for saving mel spectogram\nimport noisereduce as nr # background noise removal\nfrom sklearn.preprocessing import LabelEncoder # we have text labels\nfrom sklearn.model_selection import StratifiedKFold # cross-validation\nimport torch \nimport torchaudio # for loading audio\nfrom torch.utils.data import Dataset, DataLoader # for managing dataset. \nimport torch.nn as nn # needed for super class reference.\nimport timm # \"pyTorch Image Models\", contains pre-trained models.\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu' # GPU availability.\nfrom torch.optim import Adam # Adam Optimizer for G.D. \nfrom tqdm import tqdm # for progress bar. \nimport torch.nn.functional as F\nfrom sklearn.metrics import f1_score # metrics we need. \nfrom sklearn.metrics import accuracy_score \nfrom sklearn.metrics import classification_report\nfrom sklearn.metrics import confusion_matrix\nfrom sklearn.metrics import roc_curve, roc_auc_score\nimport seaborn as sns # templates for confusion matrix","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-11-24T13:16:34.653058Z","iopub.execute_input":"2023-11-24T13:16:34.653361Z","iopub.status.idle":"2023-11-24T13:16:44.314009Z","shell.execute_reply.started":"2023-11-24T13:16:34.653332Z","shell.execute_reply":"2023-11-24T13:16:44.313221Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---","metadata":{}},{"cell_type":"markdown","source":"# Metadata File Analysis\n#### The dataset comes with audio clips and a metadata file, which contains various information about the audio. \n#### Let's now look into the metadata file, and in next section we will take a look of audio files. ","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv('/kaggle/input/birdclef-2023/train_metadata.csv') \ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:16:44.315531Z","iopub.execute_input":"2023-11-24T13:16:44.315873Z","iopub.status.idle":"2023-11-24T13:16:44.478523Z","shell.execute_reply.started":"2023-11-24T13:16:44.315839Z","shell.execute_reply":"2023-11-24T13:16:44.477600Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f\"Number of Samples: - {len(df)}\")\nprint(f\"Number of columns: - {len(df.columns)}\")\n\ncolumns = \"\"\nfor col in df.columns:\n    columns = columns + col + \", \"\ncolumns = columns[:-2] # ignoring the last \", \" characters.\nprint(f\"Names of columns are: - {columns}\")","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:16:44.481083Z","iopub.execute_input":"2023-11-24T13:16:44.481460Z","iopub.status.idle":"2023-11-24T13:16:44.486959Z","shell.execute_reply.started":"2023-11-24T13:16:44.481426Z","shell.execute_reply":"2023-11-24T13:16:44.486080Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.isnull().sum() # checking for null values","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:16:44.488020Z","iopub.execute_input":"2023-11-24T13:16:44.488269Z","iopub.status.idle":"2023-11-24T13:16:44.517193Z","shell.execute_reply.started":"2023-11-24T13:16:44.488247Z","shell.execute_reply":"2023-11-24T13:16:44.516382Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f\"Number of classes: - {len(df.primary_label.value_counts())}\")","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:16:44.518407Z","iopub.execute_input":"2023-11-24T13:16:44.518684Z","iopub.status.idle":"2023-11-24T13:16:44.532806Z","shell.execute_reply.started":"2023-11-24T13:16:44.518659Z","shell.execute_reply":"2023-11-24T13:16:44.531948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Observations\n#### - There are 16941 bird audio samples.\n#### - There are 12 columns describing a sample.\n##### - The columns are: - primary_label, secondary_labels, type, latitude, longitude, scientific_name, common_name, author, license, rating, url and filename.\n##### > ***primary_label*** gives the target label for the classification process. It also gives the shorthand name or species for the bird.\n##### > ***type*** gives the bird sound type, for example, song, call etc.\n##### > ***filename*** gives the relative path to the audio clip. \n##### > Note! For the purpose of running a Deep Learning Classifier model the columns ***primary_label***, ***type*** and ***filename*** are sufficient. \n#### - There are 264 unique birds in the dataset.\n#### - Dataset contains some null values in columns *latitude* and *longitude*.","metadata":{}},{"cell_type":"markdown","source":"## Dataset Distributions\n#### Let's try to look at how well the dataset is distributed based on columns ***primary_label*** and ***type***.","metadata":{}},{"cell_type":"markdown","source":"### 1. Species Count Distribution\n#### This plot is based on column ***primary_label***. It will help us infer number of sample per target class.\n#### Highlight from above observations, there 264 birds, hence 264 unique values in this column. ","metadata":{}},{"cell_type":"code","source":"class_count = df.primary_label.value_counts()\n\nfig = px.bar(\n    x=class_count.values, y=class_count.index,\n    orientation='h', height=1000, \n    title=\"Species Count Distribution\"\n)\nfig.update_layout(xaxis_title=\"Count\", yaxis_title=\"Label\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:16:44.533910Z","iopub.execute_input":"2023-11-24T13:16:44.534209Z","iopub.status.idle":"2023-11-24T13:16:46.170678Z","shell.execute_reply.started":"2023-11-24T13:16:44.534184Z","shell.execute_reply":"2023-11-24T13:16:46.169812Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Observations from Species Count Distribution\n#### - There are 16941 bird audio samples.\n#### - There are 264 unique birds in the dataset.\n#### - Certain species have only 1 audio recoding and some have 500 audio recording. Remain birds have number of samples between 1 and 500. \n#### - The dataset is ***heavily imbalanced*** with regard to column ***primary_label***.","metadata":{}},{"cell_type":"markdown","source":"### 2. Sound Type Count Distribution\n#### This plot is based on column ***type***. It will help us infer number of sample per sound type.\n#### What is sound type?\n##### Sound type denotes various sounds produced by the bird in the given audio clip. \n#### What are the various sounds produced by the bird?\n##### Birds can make lot's of sound. For example they can singing ***song***, or make ***call*** to other birds, or ***peck*** some wood, or even ***wingflaps***.\n#### Let's take a look at how many such sound samples are available per ***type***. ","metadata":{}},{"cell_type":"code","source":"type_labels = sum([eval(x) for x in df.type], [])\ntype_counts = pd.value_counts(type_labels)\n\nfig = px.bar(\n    x=type_counts.values, y=[\n        f'{\" \".join(x.split(\" \")[:3])} ...' \n        if len(x.split(\" \")) > 3 else x \n        for x in type_counts.index\n    ],\n    orientation='h', height=1000,\n    title=\"Sound Type Count Distribution\"\n)\nfig.update_layout(xaxis_title=\"Count\", yaxis_title=\"Audio type\")\nfig.show()\n\nclass_count = df.primary_label.value_counts()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:16:46.171846Z","iopub.execute_input":"2023-11-24T13:16:46.172167Z","iopub.status.idle":"2023-11-24T13:16:47.524575Z","shell.execute_reply.started":"2023-11-24T13:16:46.172140Z","shell.execute_reply":"2023-11-24T13:16:47.523648Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Observations from Sound Type Count Distribution\n#### - There are 6808 samples which contain bird calls. \n#### - There are 6411 samples which contain bird songs. \n#### - Note! An audio clip can contain multiple type of bird sounds. \n#### - The dataset is ***heavily imbalanced*** with regards to column ***type***.","metadata":{}},{"cell_type":"markdown","source":"--------------------","metadata":{}},{"cell_type":"markdown","source":"# Audio Analysis","metadata":{}},{"cell_type":"markdown","source":"#### Let's us do some exploration with regards to audio clips and try to get some analysis.\n#### Let's define a helper function ***audio_analysis*** which will: play the audio, give its waveform and print its Mel Spectogram","metadata":{}},{"cell_type":"code","source":"def audio_analysis(filename, noise_remove=False):\n    \"\"\"\n    Input: \n    filename - contains the path to audio file. \n    noise_remove - should the background noise be removed. By default background noise is not removed. \n    \n    Output: - \n    Audio playback - Gives a player manager for audio. \n    Waveform \n    Mel Spectogram\n    \"\"\"\n    \n    # Loading Audio\n    samples, sample_rate = librosa.load(filename) \n    \n    # Removing Noise\n    if(noise_remove):\n        samples = nr.reduce_noise(y=samples, sr=sample_rate)\n\n    # Audio playback\n    print(\"\\nClick the play button to play the audio clip.\\n\")\n    display(Audio(samples, rate=sample_rate))\n\n    # Displaying Waveform\n    plt.figure(figsize=(10,5))\n    librosa.display.waveshow(samples, sr=sample_rate)\n    plt.title(\"Waveform Graph\")\n    \n    # Displaying Mel Spectogram\n    plt.figure(figsize=(5,4))\n    mel = librosa.feature.melspectrogram(y=samples, sr=sample_rate, n_mels=128, fmax=8000)\n    librosa.display.specshow(librosa.power_to_db(mel, ref=np.max),\n                            y_axis='mel', fmax=8000, x_axis='time')\n    plt.title('Mel Spectrogram')\n    plt.colorbar(format='%+2.0f dB')\n    \n    plt.tight_layout()\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:16:47.525996Z","iopub.execute_input":"2023-11-24T13:16:47.526644Z","iopub.status.idle":"2023-11-24T13:16:47.534660Z","shell.execute_reply.started":"2023-11-24T13:16:47.526608Z","shell.execute_reply":"2023-11-24T13:16:47.533816Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Let's try to look at a sample from bird class called ***barswa***.","metadata":{}},{"cell_type":"code","source":"df[df.filename==\"barswa/XC113914.ogg\"]","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:16:47.539298Z","iopub.execute_input":"2023-11-24T13:16:47.539604Z","iopub.status.idle":"2023-11-24T13:16:47.560581Z","shell.execute_reply.started":"2023-11-24T13:16:47.539575Z","shell.execute_reply":"2023-11-24T13:16:47.559732Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"audio_analysis(\"/kaggle/input/birdclef-2023/train_audio/barswa/XC113914.ogg\")","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:16:47.561573Z","iopub.execute_input":"2023-11-24T13:16:47.561879Z","iopub.status.idle":"2023-11-24T13:16:55.365091Z","shell.execute_reply.started":"2023-11-24T13:16:47.561842Z","shell.execute_reply":"2023-11-24T13:16:55.364222Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Observation\n#### - We can see that the above waveform is filled with background noises, evident in the audio playback. \n#### - This is serious problem. Hence let's try to remove the noise. \n","metadata":{}},{"cell_type":"code","source":"audio_analysis(\"/kaggle/input/birdclef-2023/train_audio/barswa/XC113914.ogg\", noise_remove=True)","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:16:55.366749Z","iopub.execute_input":"2023-11-24T13:16:55.367272Z","iopub.status.idle":"2023-11-24T13:16:58.242719Z","shell.execute_reply.started":"2023-11-24T13:16:55.367236Z","shell.execute_reply":"2023-11-24T13:16:58.241659Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Observation\n#### The new waveform looks much better, with regards to noise. \n#### The noise reduction is also evident from audio playback. ","metadata":{}},{"cell_type":"markdown","source":"#### Looking at another bird class called ***afpkin1***.","metadata":{}},{"cell_type":"code","source":"df[df.filename==\"afpkin1/XC704863.ogg\"]","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:16:58.244148Z","iopub.execute_input":"2023-11-24T13:16:58.244437Z","iopub.status.idle":"2023-11-24T13:16:58.261650Z","shell.execute_reply.started":"2023-11-24T13:16:58.244411Z","shell.execute_reply":"2023-11-24T13:16:58.260116Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"audio_analysis(\"/kaggle/input/birdclef-2023/train_audio/afpkin1/XC704863.ogg\")","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:16:58.263551Z","iopub.execute_input":"2023-11-24T13:16:58.263936Z","iopub.status.idle":"2023-11-24T13:16:59.350816Z","shell.execute_reply.started":"2023-11-24T13:16:58.263884Z","shell.execute_reply":"2023-11-24T13:16:59.349952Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Let's remove the background noise. ","metadata":{}},{"cell_type":"code","source":"audio_analysis(\"/kaggle/input/birdclef-2023/train_audio/afpkin1/XC704863.ogg\", noise_remove=True)","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:16:59.352219Z","iopub.execute_input":"2023-11-24T13:16:59.352823Z","iopub.status.idle":"2023-11-24T13:17:00.405746Z","shell.execute_reply.started":"2023-11-24T13:16:59.352785Z","shell.execute_reply":"2023-11-24T13:17:00.404728Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### For one last time, let's look at another bird class called ***afrgos1***.","metadata":{}},{"cell_type":"code","source":"df[df.filename==\"afrgos1/XC123946.ogg\"]","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:00.407230Z","iopub.execute_input":"2023-11-24T13:17:00.407822Z","iopub.status.idle":"2023-11-24T13:17:00.427518Z","shell.execute_reply.started":"2023-11-24T13:17:00.407786Z","shell.execute_reply":"2023-11-24T13:17:00.426645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"audio_analysis(\"/kaggle/input/birdclef-2023/train_audio/afrgos1/XC123946.ogg\")","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:00.428632Z","iopub.execute_input":"2023-11-24T13:17:00.428934Z","iopub.status.idle":"2023-11-24T13:17:01.667803Z","shell.execute_reply.started":"2023-11-24T13:17:00.428901Z","shell.execute_reply":"2023-11-24T13:17:01.666917Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"audio_analysis(\"/kaggle/input/birdclef-2023/train_audio/afrgos1/XC123946.ogg\", noise_remove=True)","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:01.668953Z","iopub.execute_input":"2023-11-24T13:17:01.669238Z","iopub.status.idle":"2023-11-24T13:17:03.309781Z","shell.execute_reply.started":"2023-11-24T13:17:01.669213Z","shell.execute_reply":"2023-11-24T13:17:03.308937Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Observations From Above Three Audio Clips\n#### - Each Audio clip is of **varying** length. \n#### - We don't know at what **timestamp** the bird makes sound. \n#### - Audio includes **noises** from other animals as well. This is can be removed to a certain extent only, beyond which nothing can be done.\n#### - Pointing out in specific, \n##### -> the first sample contains the bird sound with some random noise added, which was not removed by SOTA noise reducer(its fault)\n##### -> the second sample is the ***ideal*** one, which just contains the bird sound\n##### -> the third sample contains lot of noises, which includes other animals as well, cannot blame on SOTA noise reducer, nothing can be done :( ","metadata":{}},{"cell_type":"markdown","source":"----","metadata":{}},{"cell_type":"markdown","source":"# Issues found from Analysis\n#### 1. Some data samples contain ***null entries***. \n#### 2. The dataset is ***heavily imbalanced*** with regards to columns, *type* and *primary_label*. \n#### 3. Each audio clip is of ***varying length***, due to which we will get different length spectogram. \n#### 4. We need to make an educate crop, because we don't know the ***timestamp*** of where the bird is making the noise.\n#### 5. Deal with various kinds of ***noise*** present in the audio, since they drastically affect the spectogram. ","metadata":{}},{"cell_type":"markdown","source":"---","metadata":{}},{"cell_type":"markdown","source":"# Sampling ","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv(\"/kaggle/input/birdclef-2023/train_metadata.csv\")\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:03.310931Z","iopub.execute_input":"2023-11-24T13:17:03.311217Z","iopub.status.idle":"2023-11-24T13:17:03.401353Z","shell.execute_reply.started":"2023-11-24T13:17:03.311193Z","shell.execute_reply":"2023-11-24T13:17:03.400410Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Removing Some Columns","metadata":{}},{"cell_type":"code","source":"df = df.drop([\"secondary_labels\", \"latitude\", \"longitude\", \"common_name\", \"license\", \"rating\", \"scientific_name\", \"url\", \"author\"], axis=1)\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:03.402527Z","iopub.execute_input":"2023-11-24T13:17:03.402815Z","iopub.status.idle":"2023-11-24T13:17:03.415052Z","shell.execute_reply.started":"2023-11-24T13:17:03.402790Z","shell.execute_reply":"2023-11-24T13:17:03.414138Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Sampling on *type* column\n#### As mentioned before, the type column makes the dataset unbalanced. To fix this, let's restrict our samples to type=\"\\*calls\\*\"(denotes regex), i.e., we will only take bird samples that contains calls. \n#### Note! There are different kinds of call, to name a few, *alarm call*, *two-note call*, *call* itself, etc. We will be taking all these. ","metadata":{}},{"cell_type":"code","source":"# filtering only bird sounds of type call. \nfor i in range(len(df)):\n    if \"call\" not in df.loc[i, \"type\"]:\n        df.loc[i, \"type\"] = np.nan\n\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:03.416162Z","iopub.execute_input":"2023-11-24T13:17:03.416464Z","iopub.status.idle":"2023-11-24T13:17:04.360688Z","shell.execute_reply.started":"2023-11-24T13:17:03.416438Z","shell.execute_reply":"2023-11-24T13:17:04.359722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = df.dropna().reset_index(drop=True)\nprint(df.count())\nprint()\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:04.362050Z","iopub.execute_input":"2023-11-24T13:17:04.362360Z","iopub.status.idle":"2023-11-24T13:17:04.384426Z","shell.execute_reply.started":"2023-11-24T13:17:04.362334Z","shell.execute_reply":"2023-11-24T13:17:04.383560Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Observation\n#### The newly sampled clips, will also contain other ***type*** values like songs, adult etc. \n#### From here on will work with bird call identification alone.\nNote: We no longer the column type, hence we can remove it. ","metadata":{}},{"cell_type":"code","source":"df = df.drop(['type'], axis=1)\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:04.385499Z","iopub.execute_input":"2023-11-24T13:17:04.385768Z","iopub.status.idle":"2023-11-24T13:17:04.394724Z","shell.execute_reply.started":"2023-11-24T13:17:04.385744Z","shell.execute_reply":"2023-11-24T13:17:04.393870Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Sampling on *primary_label* column\n#### The primary_label column also makes the dataset unbalanced. \n#### To resolve this issue, we will take only few classes which contains samples greater than a certain threshold, 300.  ","metadata":{}},{"cell_type":"code","source":"# picking the birds with most samples; birds with samples > 300.\ndf.primary_label.value_counts().sort_values()[-10:]","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:04.395973Z","iopub.execute_input":"2023-11-24T13:17:04.396293Z","iopub.status.idle":"2023-11-24T13:17:04.409166Z","shell.execute_reply.started":"2023-11-24T13:17:04.396262Z","shell.execute_reply":"2023-11-24T13:17:04.408424Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# taking only the last 8 species in the sorted df. \nspecies = [\"combuz1\", \"comsan\", \"woosan\", \"eaywag1\", \"eubeat1\", \"cohmar1\", \"barswa\", \"litegr\"]\n\nfor i in range(len(df)):\n    if df.loc[i, \"primary_label\"] not in species:\n        df.loc[i, \"primary_label\"] = np.nan\n        \ndf = df.dropna().reset_index(drop=True)\nprint(f\"New samples count: - \\n{df.count()}\\n\")\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:04.410260Z","iopub.execute_input":"2023-11-24T13:17:04.410518Z","iopub.status.idle":"2023-11-24T13:17:05.154112Z","shell.execute_reply.started":"2023-11-24T13:17:04.410495Z","shell.execute_reply":"2023-11-24T13:17:05.153243Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Some classes have more than 300 samples. But, it would be best when number of samples are sample across all classes. ","metadata":{}},{"cell_type":"code","source":"combuz1 = df.loc[(df['primary_label'] == 'combuz1')]\ncomsan = df.loc[(df['primary_label'] == 'comsan')]\nwoosan = df.loc[(df['primary_label'] == 'woosan')]\neaywag1 = df.loc[(df['primary_label'] == 'eaywag1')]\neubeat1 = df.loc[(df['primary_label'] == 'eubeat1')]\ncohmar1 = df.loc[(df['primary_label'] == 'cohmar1')]\nbarswa = df.loc[(df['primary_label'] == 'barswa')]\nlitegr = df.loc[(df['primary_label'] == 'litegr')]\n\ncombuz1 = combuz1.sample(frac=1).reset_index(drop=True).head(300)\ncomsan = comsan.sample(frac=1).reset_index(drop=True).head(300)\nwoosan = woosan.sample(frac=1).reset_index(drop=True).head(300)\neaywag1 = eaywag1.sample(frac=1).reset_index(drop=True).head(300)\neubeat1 = eubeat1.sample(frac=1).reset_index(drop=True).head(300)\ncohmar1 = cohmar1.sample(frac=1).reset_index(drop=True).head(300)\nbarswa = barswa.sample(frac=1).reset_index(drop=True).head(300)\nlitegr = litegr.sample(frac=1).reset_index(drop=True).head(300)\n\ndf = pd.concat([combuz1, comsan, woosan, eaywag1, eubeat1, cohmar1, barswa, litegr], ignore_index=True, axis=0)\ndf.count()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.155398Z","iopub.execute_input":"2023-11-24T13:17:05.155789Z","iopub.status.idle":"2023-11-24T13:17:05.186021Z","shell.execute_reply.started":"2023-11-24T13:17:05.155754Z","shell.execute_reply":"2023-11-24T13:17:05.185220Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.187014Z","iopub.execute_input":"2023-11-24T13:17:05.187322Z","iopub.status.idle":"2023-11-24T13:17:05.203030Z","shell.execute_reply.started":"2023-11-24T13:17:05.187298Z","shell.execute_reply":"2023-11-24T13:17:05.202177Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Checkpoint\n#### We resolved the null values and imbalanced dataset.  ","metadata":{}},{"cell_type":"code","source":"class_count = df.primary_label.value_counts()\n\nfig = px.bar(\n    x=class_count.values, y=class_count.index,\n    orientation='h', height=400, width=600,\n    title=\"Species Count Distribution\"\n)\nfig.update_layout(xaxis_title=\"Count\", yaxis_title=\"Label\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.208782Z","iopub.execute_input":"2023-11-24T13:17:05.209499Z","iopub.status.idle":"2023-11-24T13:17:05.275150Z","shell.execute_reply.started":"2023-11-24T13:17:05.209455Z","shell.execute_reply":"2023-11-24T13:17:05.274142Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Now, the labels are given as string, but AI models only work with numbers, hence we will have to encode them. ","metadata":{}},{"cell_type":"code","source":"# adding a new column which contains the numeric labels. \nencoder = LabelEncoder()\ndf['primary_label_encoded'] = encoder.fit_transform(df['primary_label'])\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.276376Z","iopub.execute_input":"2023-11-24T13:17:05.276729Z","iopub.status.idle":"2023-11-24T13:17:05.287761Z","shell.execute_reply.started":"2023-11-24T13:17:05.276695Z","shell.execute_reply":"2023-11-24T13:17:05.286942Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Num_classes\nprint(f\"Number of classes: {len(df.primary_label_encoded.unique())}\")\nprint(f\"Classes are: - \\n{df.primary_label.unique()}\")","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.289023Z","iopub.execute_input":"2023-11-24T13:17:05.289312Z","iopub.status.idle":"2023-11-24T13:17:05.301234Z","shell.execute_reply.started":"2023-11-24T13:17:05.289288Z","shell.execute_reply":"2023-11-24T13:17:05.300357Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Test-Train Split","metadata":{}},{"cell_type":"code","source":"# shuffling the dataframe. \ndf = df.sample(frac=1).reset_index(drop=True)\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.302439Z","iopub.execute_input":"2023-11-24T13:17:05.302701Z","iopub.status.idle":"2023-11-24T13:17:05.317387Z","shell.execute_reply.started":"2023-11-24T13:17:05.302678Z","shell.execute_reply":"2023-11-24T13:17:05.316442Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# splitting training and testing datasets. \ndf_total = df\ndf_training = df_total[:2100] # split being 80:20 for training:testing\ndf_testing = df_total[2100:]\ndf = df_training","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.318298Z","iopub.execute_input":"2023-11-24T13:17:05.318537Z","iopub.status.idle":"2023-11-24T13:17:05.328098Z","shell.execute_reply.started":"2023-11-24T13:17:05.318516Z","shell.execute_reply":"2023-11-24T13:17:05.327391Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### K-fold cross validation","metadata":{}},{"cell_type":"code","source":"skf = StratifiedKFold(n_splits=5)\nfor k, (_, val_ind) in enumerate(skf.split(X=df, y=df['primary_label_encoded'])):\n    df.loc[val_ind, 'kfold'] = k\n\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.329064Z","iopub.execute_input":"2023-11-24T13:17:05.329358Z","iopub.status.idle":"2023-11-24T13:17:05.350858Z","shell.execute_reply.started":"2023-11-24T13:17:05.329335Z","shell.execute_reply":"2023-11-24T13:17:05.349927Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# shuffling the dataset\ndf = df.sample(frac=1).reset_index(drop=True)\ndf","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.351860Z","iopub.execute_input":"2023-11-24T13:17:05.352201Z","iopub.status.idle":"2023-11-24T13:17:05.367475Z","shell.execute_reply.started":"2023-11-24T13:17:05.352169Z","shell.execute_reply":"2023-11-24T13:17:05.366643Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Configurations","metadata":{}},{"cell_type":"code","source":"CONFIG = {\"epochs\": 10, # n passes over the entire data.\n          \"num_classes\": 8, # number of unique bird species. \n          \"batch_size\": 8, # update the parameters after running through n samples.\n          \"sample_rate\": 32_000, # high sampling rate => high resolution \n          \"hop_length\": 512, # distance between consecutive FFTs.\n          \"n_mels\": 224, # number of mel bins. Frequency(y-axis) is split into mel bins. \n          \"n_fft\": 1024, # number of data points used in each discrete Fourier transform.\n          \"pad_length\": 4, # amount of empty sound to be added for one end, keep it as integers. \n          \"crop_length\": 4 # length of audio to crop for one end, keep it as integers. \n         }\n# \n# Note!\n# pad length and crop length are from a point and along a direction. \n# Eg. let audio clip be 25seconds, and let pad_length be 5 seconds. We then pad 5 seconds \n# on either end of clip, hence audio becomes 35seconds long.\n# Say, we a pick a timestep 20 and our crop_length = 5. Now our window becomes [15,25], \n# we cropped 5 seconds from either side. ","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.368586Z","iopub.execute_input":"2023-11-24T13:17:05.369130Z","iopub.status.idle":"2023-11-24T13:17:05.374617Z","shell.execute_reply.started":"2023-11-24T13:17:05.369097Z","shell.execute_reply":"2023-11-24T13:17:05.373773Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Audio Processing for Model\n## Operations Performed\n#### 1. Load the audio. \n#### 2. Convert the audio from stereo to mono.\n#### 3. Resample it to target sample rate. \n#### 4. Pad the audio. \n#### 5. Crop the audio. \n#### 6. Remove background noise.\n#### 7. Convert to mel spectogram. \n#### 8. Convert to RGB. \n#### 9. Get the label of image.\n\n## FAQ on Audio Processing\n### Why should we convert from stereo to mono?\nFor consistency, compatibility and less resource consumption.\n\n### Why should we resample the image?\nSample rate means number of sample per second. This affects spectogram. All audio clips should have same sample rate for consistency.\n\n### Why pad and crop the audio?\nDifferent samples have different audio length, which affects the size of spectogram. And CNN architecture require fixed size input. This requires all the samples to have same length. In our case we will have the legth as 10seconds(crop_length * 2). But we need to address the corner case where the length of the audio clip is less than window size. Hence we pad pad_length seconds to either ends of the audio clip. \n\n### How to crop the audio?\nWe dicussed this issue above, saying we don't know where to crop the audio. Since the bird sound can be present at any timestamp. To overcome this issue we have to make an assumption that, in the clip bird's sound will have the amplitude, and its neighbourhood will have more of its sound, hence take crop_length seconds from left and right of this point as sample. And it is also certain that first and last 5 seconds of the audio will not contain this peak amplitude, since we padded 0tensors at either end, hence when cropping we never run into a case where we go beyond either ends of audio clip.  ","metadata":{}},{"cell_type":"code","source":"class BirdCLEFDataset(Dataset):\n    def __init__(self, df, target_sample_rate=CONFIG[\"sample_rate\"], max_time=5, image_transforms=None):\n        \n        self.file_paths = df['filename'].values # holds the path for audio.  \n        self.labels = df['primary_label_encoded'].values # holds the label of that audio. \n        self.target_sample_rate = target_sample_rate\n        self.image_transforms = image_transforms # if any transform was provided then use it. \n        \n    def __len__(self):\n        return len(self.file_paths) \n    \n    def __getitem__(self, index): \n        \"\"\"\n        Input - index of element. \n        Output - image, label. \n        \"\"\"\n        \n        # 1. Load the audio\n        filepath = f'/kaggle/input/birdclef-2023/train_audio/{self.file_paths[index]}'\n        audio, sample_rate = torchaudio.load(filepath)\n        \n        \n        # 2. Convert the audio from stereo to mono\n        audio = self.to_mono(audio) \n        \n        \n        # 3. Resample it to target sample rate\n        if sample_rate != self.target_sample_rate:\n            resample = Resample(sample_rate, self.target_sample_rate)\n            audio = resample(audio)\n        \n        \n        # 4. Pad the audio. \n        audio = self.pad_audio(audio)\n        \n        \n        # 5. Crop the audio.\n        audio = self.crop_audio(audio)\n\n        \n        # 6. Remove background noise\n        audio = torch.from_numpy(nr.reduce_noise(y=audio, sr=self.target_sample_rate))\n\n        \n        # 7. Convert to Mel Spectogram\n        mel_spectogram = torchaudio.transforms.MelSpectrogram(sample_rate=self.target_sample_rate, \n                                        n_mels=CONFIG['n_mels'], \n                                        n_fft=CONFIG['n_fft'])\n        mel = mel_spectogram(audio)\n        \n        \n        # 8. Convert image to RGB\n        image = torch.stack([mel, mel, mel]) \n        \n        # 9. Get the label for image\n        label = torch.tensor(self.labels[index]) \n        \n        return image, label\n    \n    \n    def pad_audio(self, audio): \n        \n        pad_length = CONFIG[\"pad_length\"]\n        pad_sample = self.target_sample_rate * pad_length\n        pad_tensor = torch.zeros(pad_sample)\n        \n        # padding both ends of audio with 0s.\n        audio = torch.cat((pad_tensor, audio, pad_tensor))\n        return audio\n        \n        \n    def crop_audio(self, audio):\n        \n        crop_length = CONFIG[\"crop_length\"]\n        crop_sample = self.target_sample_rate * crop_length\n        \n        # finding the peak amplitude\n        peak_index = torch.argmax(audio).item()\n        \n        # cropping 5seconds (left and right) from that point\n        return audio[peak_index-crop_sample : peak_index+crop_sample] \n        \n        \n    def to_mono(self, audio):\n        return torch.mean(audio, axis=0) ","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.375838Z","iopub.execute_input":"2023-11-24T13:17:05.376179Z","iopub.status.idle":"2023-11-24T13:17:05.389973Z","shell.execute_reply.started":"2023-11-24T13:17:05.376148Z","shell.execute_reply":"2023-11-24T13:17:05.389085Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Definition","metadata":{}},{"cell_type":"markdown","source":"## Model used is ***EfficientNet***.\n### Why CNN architecutre over RNN architecture?\n#### > The given audio data is naturally represented in time-frequency making it suitable for spectogram representation.\n#### > The current task of classification is simple, hence a well designed CNN would suffice, RNN outperforms CNN only when there are long-term dependencies. \n#### > Computationlly CNNs better than RNNs. \n##### ***CNNs have proven to be highly effective for extracting spatial features from audio signals, making them suitable for tasks such as music genre classification and environmental sound classification. The remarkable success of CNNs in this field is due to their ability to capture high-level features from the audio data. RNNs are particularly well-suited for tasks that require temporal dependencies, such as speech classification and audio-sequence classification.*** <br>-[K. Zaman et al.: Survey of Audio Classification Using Deep Learning](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10258355)\n\n\n\n\n### Why EfficientNet?\n#### The below plots will explain it all. \n![](https://raw.githubusercontent.com/tensorflow/tpu/master/models/official/efficientnet/g3doc/flops.png)\nFLOPS - Floating Point Operations Per Second.\n\n---\n![](https://raw.githubusercontent.com/tensorflow/tpu/master/models/official/efficientnet/g3doc/params.png)\n\n#### It clearly outperforms all other popular classification models. And, it has fewer computations than others. \n#### Efficientnet models are also very suitable for wild dataset for the reasons mentioned above.\nThe term \"wild dataset\" in the context of deep learning generally refers to datasets that are collected from real-world, diverse, and uncontrolled environments. These datasets are often contrasted with \"controlled\" datasets that are carefully curated or designed under specific conditions.\n\n#### BirdCLEF 2023 dataset used in this project is a perfect example of \"wild dataset\". \n\n#### From the above we can infer that we have many models of EfficientNet.\n","metadata":{}},{"cell_type":"code","source":"class GeM(nn.Module): # defines Generalized Mean Pooling\n    def __init__(self, p=3, eps=1e-6): \n        \"\"\"\n        'p', hyperparameter that determines the degree of pooling in the GeM Pooling\n\n        eps is the smallest representable number such that 1.0 + eps != 1.0\n        The difference between 1.0 and the next smallest representable float\n        \"\"\"\n        super(GeM, self).__init__()\n        self.p = nn.Parameter(torch.ones(1)*p)\n        self.eps = eps \n        \n        \n    def forward(self, x):\n        \"\"\" Calls the pooling function. \"\"\"\n        return self.gem(x, p=self.p, eps=self.eps)\n        \n        \n    def gem(self, x, p=3, eps=1e-6):\n        \"\"\" \n        Clamp ensures that input tensor 'x' is non-negative. \n        Pow(p) is element wise power. \n        \"\"\"\n        return F.avg_pool2d(x.clamp(min=eps).pow(p), (x.size(-2), x.size(-1))).pow(1./p)\n        \n        \n    def __repr__(self):\n        \"\"\" Returns the string representation of the class. \"\"\"\n        return self.__class__.__name__ + \\\n                '(' + 'p=' + '{:.4f}'.format(self.p.data.tolist()[0]) + \\\n                ', ' + 'eps=' + str(self.eps) + ')'\n    \n    \nclass BirdCLEFModel(nn.Module): \n    def __init__(self, model_name=\"tf_efficientnet_b5_ns\", embedding_size=768, pretrained=True):\n        super(BirdCLEFModel, self).__init__()\n        \n        # getting a pretrained model\n        self.model = timm.create_model(model_name, pretrained=pretrained)\n        \n        # in_feature size is set to default. \n        in_features = self.model.classifier.in_features \n        \n        # we will define our own FC and pooling layer. \n        self.model.classifier = nn.Identity() \n        self.model.global_pool = nn.Identity()\n        \n        # defining the pooling and FC layers. \n        self.pooling = GeM()\n        self.embedding = nn.Linear(in_features, embedding_size) \n        self.fc = nn.Linear(embedding_size, CONFIG['num_classes'])\n\n    def forward(self, images):\n        \n        # getting the abstract representations. \n        features = self.model(images) \n        \n        # flattening the representations. \n        pooled_features = self.pooling(features).flatten(1) \n        \n        # passing it to embedding layer. \n        embedding = self.embedding(pooled_features) \n        \n        # getting the target output.\n        output = self.fc(embedding)\n        \n        return output","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.391120Z","iopub.execute_input":"2023-11-24T13:17:05.391391Z","iopub.status.idle":"2023-11-24T13:17:05.406245Z","shell.execute_reply.started":"2023-11-24T13:17:05.391361Z","shell.execute_reply":"2023-11-24T13:17:05.405381Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Loss Function\n#### This is a multi-class classification, hence cross-entropy.","metadata":{}},{"cell_type":"code","source":"def loss_fn(outputs, labels):\n    return nn.CrossEntropyLoss()(outputs, labels)","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.407148Z","iopub.execute_input":"2023-11-24T13:17:05.407386Z","iopub.status.idle":"2023-11-24T13:17:05.421500Z","shell.execute_reply.started":"2023-11-24T13:17:05.407365Z","shell.execute_reply":"2023-11-24T13:17:05.420583Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train, Validation and Test","metadata":{}},{"cell_type":"markdown","source":"## Train","metadata":{}},{"cell_type":"code","source":"def train(model, data_loader, optimizer, scheduler, device, epoch):\n    \"\"\"\n    Inputs: - \n    model: Our neural network model.\n    data_loader: Provides batches of training data.\n    optimizer: Algorithm for updating the model's parameters.\n    scheduler: For adjusting the learning rate during training.\n    device: For GPU availability.\n    epoch: The current epoch number.\n    \n    This method is executed for each epoch, i.e., one run of dataset. \n    \"\"\"\n    model.train()\n    running_loss = 0\n    \n    # for progress bar and also returns a batch everytime it invoked.\n    loop = tqdm(data_loader, position=0)\n    \n    for i, (mels, labels) in enumerate(loop):\n        # running through samples as per batch size. \n        # number of samples per look is given by batch size.\n        \n        # moving mels and labels to GPU to perform computation.\n        mels = mels.to(device)  \n        labels = labels.to(device) \n        \n        # getting the predictions.\n        outputs = model(mels) \n        _, preds = torch.max(outputs, 1)\n        \n        # finding current loss using cross-entropy\n        loss = loss_fn(outputs, labels)\n        \n        loss.backward() # computes gradients. \n        optimizer.step() # update the model. \n        optimizer.zero_grad() # clear the gradients. \n         \n        if scheduler is not None:\n            scheduler.step() # updates learning rate.\n            \n        running_loss += loss.item()\n        \n        # Progress bar contents.\n        loop.set_description(f\"Epoch [{epoch+1}/{CONFIG['epochs']}]\")\n        loop.set_postfix(loss=loss.item())\n\n    return running_loss/len(data_loader) # normalized loss.","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.422655Z","iopub.execute_input":"2023-11-24T13:17:05.423001Z","iopub.status.idle":"2023-11-24T13:17:05.432779Z","shell.execute_reply.started":"2023-11-24T13:17:05.422968Z","shell.execute_reply":"2023-11-24T13:17:05.431873Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Validation","metadata":{}},{"cell_type":"code","source":"def valid(model, data_loader, device, epoch):\n    \"\"\"\n    Inputs: - \n    model: Our neural network model.\n    data_loader: Provides batches of training data.\n    device: For GPU availability.\n    epoch: The current epoch number.\n    \n    This method is executed for each epoch, i.e., one run of dataset. \n    \"\"\"\n    model.eval()\n    \n    running_loss = 0\n    pred = []\n    label = []\n    \n    # for progress bar and also returns a batch everytime it invoked.\n    loop = tqdm(data_loader, position=0)\n    \n    for mels, labels in loop:\n        # running through samples as per batch size. \n        # number of samples per look is given by batch size.\n        \n        # moving mels and labels to GPU to perform computation.\n        mels = mels.to(device)\n        labels = labels.to(device)\n        \n        # getting the predictions.\n        outputs = model(mels)\n        _, preds = torch.max(outputs, 1)\n        \n        # finding current loss using cross-entropy\n        loss = loss_fn(outputs, labels)\n            \n        running_loss += loss.item()\n        \n        \"\"\"\n        pred(label) - a list containing predictions(label).\n        preds(labels) - a tensor containing predictions(labels).\n        view(-1) - flatten the tensor.\n        cpu() - moves to CPU if on a GPU.\n        numpy() - converts the tensor to a NumPy array.\n        extend() - appending elements to list.\n        \"\"\"\n        pred.extend(preds.view(-1).cpu().detach().numpy())\n        label.extend(labels.view(-1).cpu().detach().numpy())\n        \n        # Progress bar contents.\n        loop.set_description(f\"Epoch [{epoch+1}/{CONFIG['epochs']}]\")\n        loop.set_postfix(loss=loss.item())\n        \n    valid_f1 = f1_score(label, pred, average='macro')\n    valid_accuracy = accuracy_score(label, pred)\n    print(f\"\\nValidation Accuracy: {valid_accuracy}\")\n    \n    return running_loss/len(data_loader), valid_f1","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.433981Z","iopub.execute_input":"2023-11-24T13:17:05.434243Z","iopub.status.idle":"2023-11-24T13:17:05.447635Z","shell.execute_reply.started":"2023-11-24T13:17:05.434220Z","shell.execute_reply":"2023-11-24T13:17:05.446801Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Testing","metadata":{}},{"cell_type":"code","source":"def print_confusion_matrix(y_test, y_prediction, labels=species):\n    cm = confusion_matrix(y_test, y_prediction)\n    plt.figure(figsize=(10,10))\n    sns.heatmap(cm, annot=True, fmt=\".0f\", linewidths=.5, square = True, cmap = 'Blues')\n    plt.ylabel('Actual label')\n    plt.xlabel('Predicted label')\n    plt.xticks(np.arange(8)+0.5, labels)\n    plt.yticks(np.arange(8)+0.5, labels)\n    plt.show()\n    print()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.448835Z","iopub.execute_input":"2023-11-24T13:17:05.449420Z","iopub.status.idle":"2023-11-24T13:17:05.462135Z","shell.execute_reply.started":"2023-11-24T13:17:05.449388Z","shell.execute_reply":"2023-11-24T13:17:05.461387Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def test(model, data_loader, device):\n    \"\"\"\n    Inputs: - \n    model: Our neural network model.\n    data_loader: Provides batches of training data.\n    device: For GPU availability.\n    \"\"\"\n    model.eval()\n    \n    pred = []\n    label = []\n    \n    loop = tqdm(data_loader, position=0)\n    for mels, labels in loop:\n        \n        # moving mels and labels to GPU to perform computation.\n        mels = mels.to(device)\n        labels = labels.to(device)\n        \n        # getting the predictions.\n        outputs = model(mels)\n        \n        # outputs_detach contains raw predictions.\n        outputs_detach = outputs.detach()\n        \n        # torch.max returns max value and its index along a dimension.\n        _, preds = torch.max(outputs_detach, 1)\n        \n        \n        \"\"\"\n        pred(label) - a list containing predictions(label).\n        preds(labels) - a tensor containing predictions(labels).\n        view(-1) - flatten the tensor.\n        cpu() - moves to CPU if on a GPU.\n        numpy() - converts the tensor to a NumPy array.\n        extend() - appending elements to list.\n        \"\"\"\n        pred.extend(preds.view(-1).cpu().numpy())\n        label.extend(labels.view(-1).cpu().numpy())\n    \n    species = [\"combuz1\", \"comsan\", \"woosan\", \"eaywag1\", \"eubeat1\", \"cohmar1\", \"barswa\", \"litegr\"]\n    print(\"\\n\\nTesting Report: -\\n\")\n    print(classification_report(label, pred, target_names=species))\n    print_confusion_matrix(label, pred)","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.463257Z","iopub.execute_input":"2023-11-24T13:17:05.463827Z","iopub.status.idle":"2023-11-24T13:17:05.476813Z","shell.execute_reply.started":"2023-11-24T13:17:05.463794Z","shell.execute_reply":"2023-11-24T13:17:05.476067Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Loading Data","metadata":{}},{"cell_type":"code","source":"def prepare_loaders(df, fold):\n    \n    # Split into training and validation sets based on the fold index\n    df_train = df[df.kfold != fold].reset_index(drop=True)\n    df_valid = df[df.kfold == fold].reset_index(drop=True)\n    \n    # Create BirdCLEFDataset objects for training and validation\n    train_dataset = BirdCLEFDataset(df_train, target_sample_rate=CONFIG['sample_rate'])\n    valid_dataset = BirdCLEFDataset(df_valid, target_sample_rate=CONFIG['sample_rate'])\n    \n    # Create DataLoader objects for training and validation\n    # num_workers=2 is for parallel loading\n    # dataloader shuffles the data before training, but not testing\n    # drop_last - if last batch size is less than specified batch_size, just drop it.\n    train_loader = DataLoader(train_dataset, batch_size=CONFIG['batch_size'], \n                              num_workers=2, shuffle=True, pin_memory=True, drop_last=True)\n    valid_loader = DataLoader(valid_dataset, batch_size=CONFIG['batch_size'], \n                              num_workers=2, shuffle=False, pin_memory=True)\n    \n    return train_loader, valid_loader","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.477773Z","iopub.execute_input":"2023-11-24T13:17:05.478054Z","iopub.status.idle":"2023-11-24T13:17:05.488653Z","shell.execute_reply.started":"2023-11-24T13:17:05.478030Z","shell.execute_reply":"2023-11-24T13:17:05.487881Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Training","metadata":{}},{"cell_type":"code","source":"# pushing the model to GPU. \nmodel = BirdCLEFModel().to(device)\n\n# Getting Adam Optimizer. \noptimizer = Adam(model.parameters(), lr=1e-4)\nscheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, eta_min=1e-5, T_max=10)\n# print(model)","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:05.489615Z","iopub.execute_input":"2023-11-24T13:17:05.490012Z","iopub.status.idle":"2023-11-24T13:17:10.044213Z","shell.execute_reply.started":"2023-11-24T13:17:05.489980Z","shell.execute_reply":"2023-11-24T13:17:10.043270Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# creating variables for plotting graphs. \ntrain_losses = []\nvalid_losses = []\nvalid_f1_scores = []\n\nbest_valid_f1 = 0\n\n# loading dataset with fold 0.\ntrain_loader, valid_loader = prepare_loaders(df, 0)\n\nfor epoch in range(CONFIG['epochs']):\n    train_loss = train(model, train_loader, optimizer, scheduler, device, epoch)\n    valid_loss, valid_f1 = valid(model, valid_loader, device, epoch)\n    \n    # appending the values for plotting graph.\n    train_losses.append(train_loss)\n    valid_losses.append(valid_loss)\n    valid_f1_scores.append(valid_f1)\n    \n    if valid_f1 > best_valid_f1:\n        print(f\"Validation F1 Improved: {best_valid_f1} ---> {valid_f1}\")\n        best_valid_f1 = valid_f1\n    print(\"-------------------------------------------------\\n\")\n","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:17:10.045540Z","iopub.execute_input":"2023-11-24T13:17:10.045924Z","iopub.status.idle":"2023-11-24T13:47:51.706506Z","shell.execute_reply.started":"2023-11-24T13:17:10.045873Z","shell.execute_reply":"2023-11-24T13:47:51.705326Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training Performance","metadata":{}},{"cell_type":"code","source":"# Plot the training and validation metrics\nplt.figure(figsize=(10, 5))\nplt.subplot(1, 2, 1)\nplt.plot(np.arange(CONFIG['epochs']), train_losses, label='Train Loss')\nplt.plot(np.arange(CONFIG['epochs']), valid_losses, label='Valid Loss')\nplt.title('Training and Validation Loss')\nplt.xlabel('Epoch')\nplt.ylabel('Loss')\nplt.legend()\n\nplt.subplot(1, 2, 2)\nplt.plot(np.arange(CONFIG['epochs']), valid_f1_scores, label='Valid F1 Score', color='orange')\nplt.title('Validation F1 Score')\nplt.xlabel('Epoch')\nplt.ylabel('F1 Score')\nplt.legend()\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:47:51.710197Z","iopub.execute_input":"2023-11-24T13:47:51.710502Z","iopub.status.idle":"2023-11-24T13:47:52.322677Z","shell.execute_reply.started":"2023-11-24T13:47:51.710471Z","shell.execute_reply":"2023-11-24T13:47:52.321371Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Testing","metadata":{}},{"cell_type":"code","source":"test_dataset = BirdCLEFDataset(df_testing, target_sample_rate=CONFIG['sample_rate'])\n    \ntest_loader = DataLoader(test_dataset, batch_size=CONFIG['batch_size'],num_workers=2, \n                         shuffle=True, pin_memory=True, drop_last=True)\n\ntest(model, test_loader, device)","metadata":{"execution":{"iopub.status.busy":"2023-11-24T13:47:52.323870Z","iopub.execute_input":"2023-11-24T13:47:52.324164Z","iopub.status.idle":"2023-11-24T13:48:26.455201Z","shell.execute_reply.started":"2023-11-24T13:47:52.324137Z","shell.execute_reply":"2023-11-24T13:48:26.454408Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}