{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.7.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":21669,"databundleVersionId":1692278,"sourceType":"competition"}],"dockerImageVersionId":30028,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

Rainforest Connection Species Audio Detection

\n

Automate the detection of bird and frog species in a tropical soundscape

","metadata":{}},{"cell_type":"markdown","source":"# About the Organizers","metadata":{}},{"cell_type":"markdown","source":"- **Rainforest Connection (RFCx)** created the world’s first scalable, real-time monitoring system for protecting and studying remote ecosystems. \n- Unlike visual-based tracking systems like drones or satellites, RFCx relies on acoustic sensors that monitor the ecosystem soundscape at selected locations year round. \n- RFCx technology has advanced to support a comprehensive biodiversity monitoring program that allows local partners to measure progress of wildlife restoration and recovery through principles of adaptive management. \n- The RFCx monitoring platform also has the capacity to create *convolutional neural network (CNN)* models for analysis.\n- More about them can be found [here](https://www.rfcx.org/)","metadata":{}},{"cell_type":"markdown","source":"# 1. Brief Description\n\n- The presence of rainforest species is a good indicator of the impact of climate change and habitat loss. As it's easier to hear these species than see them, it’s important to use acoustic technologies that can work on a global scale. Real-time information, such as provided through machine learning techniques, could enable **early-stage detection of human impacts** on the environment. This result could drive more effective conservation management decisions.\n- Traditional methods of assessing the diversity and abundance of species are costly and limited in space and time. And while **automatic acoustic identification via deep learning** has been successful, models require a large number of training samples per species. This limits applicability to ***rarer species***, which are central to conservation efforts. Thus, methods to *automate high-accuracy species detection in noisy soundscapes with limited training data* are the solution.","metadata":{}},{"cell_type":"markdown","source":"# 1.1 About the Competition\n\n- In this competition, you’ll automate the detection of bird and frog species in tropical soundscape recordings. You'll create your models with **limited, acoustically complex training data**. \n- Rich in more than bird and frog noises, expect to hear an insect or two, which your model will need to filter out.\n- The resulting real-time information could enable earlier detection of human environmental impacts, making environmental conservation more swift and effective.","metadata":{}},{"cell_type":"markdown","source":"Q: Oh boy, who reads all that? These are mentioned on the competition page too. Ohkay :( Let's talk in layman's terms. \nA: We need to detect audio of some number of species(rare, yeah) in the given audio recordings. \nQ: What will happen if we will do so? \nA: It will help the organizers in some way to efficiently process conservation proceedings for such rare species. \nQ: Well, tell us about the data then. \n> We need to predict the probability of all the species present in each test audio file. Some test audio files contain a single species while others contain multiple. The predictions are to be done at the audio file level, i.e., no start/end timestamps are required.\n\nQ: Oh man, why can't you speak enlgish, always quoting organizers, huh? \nA: Well, speaking of that, I got you covered [here](https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/200757). Lazy to click on the link, I got it covered below :)","metadata":{}},{"cell_type":"markdown","source":"# 1.2 Submission Formats\n\n We have got 24 classes and some 9000 training samples in total. No, this much data is not sufficient to understand the **Label weighted LRAP (label ranking average precision)** metric. First, have a look at the below data chunk. \n\n| | recording_id |\tspecies_id | songtype_id | t_min | f_min | t_max | f_max |\n|---|-------------- |---------------|-------------|--------|----------|--------|-----------| \n| 0 | 00204008d | 21 | 1 |13.8400 |3281.2500 | 14.9333| 4125.0000 |\n| 1 | 00204008d | 8\t | 1\t |24.4960 |3750.0000 | 28.6187| 5531.2500 |\n| 2 | 00204008d | 4 | 1 |15.0027 |2343.7500 | 6.8587 | 4218.7500 | \n\n\nWhat did you see? Well, if it didn't catch your attention, Let me explain as per my understanding. For single `recording id` ***00204008d***, we have got 3 species ids in that, i.e. 21, 8, and 4. What does that mean? The recording, which means the audio sample having this mentioned id, has audio of species 24, species 8, and species 4. Hence, our ground truth for this data sample would be:\n> [0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]. \n\nBut what we need to predict? We need to predict the probability of the presence of each class in this audio sample. Hence our (hypothetical) predicted vector for this sample would be:\n> [0.01, 0.01, 0.01, 0.84, 0.01, 0.01, 0.01, 0.91, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.78]. \n\nSo, yeah, it's a **multilabel classification** problem. \n\nNow quoting the submission format from the competition page:\n```\nrecording_id,s0,...,s23\n000316da7,0.1,....,0.3\n003bc2cb2,0.0,...,0.8\n...\n```\n\nAnd we need to do this for what, 1992 samples? Okay, that being said, let's worry about the evaluation metric.","metadata":{}},{"cell_type":"markdown","source":"# 1.3 Evaluation Metric","metadata":{}},{"cell_type":"markdown","source":"- The competition metric is the label-weighted [label-ranking average precision](https://scikit-learn.org/stable/modules/model_evaluation.html#label-ranking-average-precision), which is a generalization of the mean reciprocal rank measure for the case where there can be multiple true labels per test item.\n\n- The **label-weighted** part means that the overall score is the average over all the labels in the test set, where each label receives equal weight (by contrast, plain LRAP gives each test observation equal weight, thereby discounting the contribution of individual labels when when an observation has multiple labels). \n- In other words, each test observation is weighted by the number of ground truth labels found in the observation.","metadata":{}},{"cell_type":"markdown","source":"# 1.3.1 More on the Evaluation Metrics","metadata":{}},{"cell_type":"markdown","source":"**Label ranking average precision (LRAP)** averages over the samples the answer to the following question: `for each ground truth label, what fraction of higher-ranked labels were true labels?`\n\n- This performance measure will be higher if you are able to give a better rank to the labels associated with each sample. \n\n> The obtained score is always strictly greater than 0, and the best value is 1. \n\n- If there is exactly one relevant label per sample, **label ranking average precision is equivalent to the mean reciprocal rank.**\n\nFormally, given a binary indicator matrix of the ground truth labels $y \\in \\left\\{0, 1\\right\\}^{n_\\text{samples} \\times n_\\text{labels}}$ and the score associated with each label $\\hat{f} \\in \\mathbb{R}^{n_\\text{samples} \\times n_\\text{labels}}$, the average precision is defined as \n\n$LRAP(y, \\hat{f}) = \\frac{1}{n_{\\text{samples}}}\n \\sum_{i=0}^{n_{\\text{samples}} - 1} \\frac{1}{||y_i||_0}\n \\sum_{j:y_{ij} = 1} \\frac{|\\mathcal{L}_{ij}|}{\\text{rank}_{ij}}$\n\nwhere $\\mathcal{L}_{ij} = \\left\\{k: y_{ik} = 1, \\hat{f}_{ik} \\geq \\hat{f}_{ij} \\right\\}$, $\\text{rank}_{ij} = \\left|\\left\\{k: \\hat{f}_{ik} \\geq \\hat{f}_{ij} \\right\\}\\right|$, computes the cardinality of the set (i.e., the number of elements in the set), and $||\\cdot||_0$ is the $\\ell_0$ “norm” (which computes the number of nonzero elements in a vector).\n","metadata":{}},{"cell_type":"markdown","source":"

Don't forget to upvote if you like this notebook. And let me know your thoughts and findings in the comments.

","metadata":{}},{"cell_type":"markdown","source":"# 2. Deep Dive into Data","metadata":{}},{"cell_type":"markdown","source":"> In this competition, you are given audio files that include sounds from numerous species. Your task is, for each test audio file, to predict the probability that each of the given species is audible in the audio clip. While the training files contain both the species identification as well as the time the species was heard, the time localization is not part of the test predictions.\n\nAgain quoted the organizers, but that's what we have to follow :) So yeah, I hope the first part is clear, or you can refer to the section 1.2 and 1.3 of this notebook for better understanding about the format and evaluation metrics. Cool? Let's understand the last sentence. C'mon, read again.\nIn the training files (means audio recordings), we do have the timestamps information where the particular species is heard. Clear? But for test samples, we don't have that data. Well, we will use it for better insgihts later ;)\n\n***Note that the training data also includes false positive label occurrences to assist with training.***\n\nLet's look at the files and its description.\n\n# 2.1 Files\n\n- **train_tp.csv** - training data of true positive species labels, with corresponding time localization\n- **train_fp.csv** - training data of false positives species labels, with corresponding time localization\n- **sample_submission.csv** - a sample submission file in the correct format; note each species column has an s prefix.\n- **train/** - the training audio files\n- **test/** - the test audio files; the task is to predict the species found in each audio file\n- **tfrecords/{train,test}** - competition data in the TFRecord format, which includes **recording_id**, **audio_wav** (encoded in 16-bit PCM format), and **label_info** (for train only), which provides a `,` -delimited string of the columns below (minus recording_id), where multiple labels for a recording_id are `;` -delimited.\n\n# 2.1.1 Columns\n- **recording_id** - unique identifier for recording\n- **species_id** - unique identifier for species\n- **songtype_id** - unique identifier for songtype\n- **t_min** - start second of annotated signal\n- **f_min** - lower frequency of annotated signal\n- **t_max** - end second of annotated signal\n- **f_max** - upper frequency of annotated signal\n- **is_tp** - [tfrecords only] an indicator of whether the label is from the train_tp (1) or train_fp (0) file.","metadata":{}},{"cell_type":"markdown","source":"# 2.2 Imports","metadata":{}},{"cell_type":"code","source":"import gc\nimport os\nimport random\nimport numpy as np\nimport pandas as pd\nfrom tqdm import tqdm\n\nimport seaborn as sns\nimport IPython.display as ipD\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as ptc\n\nimport tensorflow as tf\n\nfrom sklearn.preprocessing import StandardScaler, MinMaxScaler\n\nmm = MinMaxScaler()\nss = StandardScaler()\n\nimport librosa\nimport librosa.display as LD\n\n%matplotlib inline","metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","execution":{"iopub.status.busy":"2023-12-15T15:42:09.582518Z","iopub.execute_input":"2023-12-15T15:42:09.583327Z","iopub.status.idle":"2023-12-15T15:42:17.794076Z","shell.execute_reply.started":"2023-12-15T15:42:09.583271Z","shell.execute_reply":"2023-12-15T15:42:17.792973Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_folder = \"../input/rfcx-species-audio-detection/test\"\ntfrecords = \"../input/rfcx-species-audio-detection/tfrecords\"\ntrain_folder = \"../input/rfcx-species-audio-detection/train\"\nsample_submission = \"../input/rfcx-species-audio-detection/sample_submission.csv\"\ntrain_tp = \"../input/rfcx-species-audio-detection/train_tp.csv\"\ntrain_fp = \"../input/rfcx-species-audio-detection/train_fp.csv\"","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:17.796464Z","iopub.execute_input":"2023-12-15T15:42:17.796815Z","iopub.status.idle":"2023-12-15T15:42:17.802425Z","shell.execute_reply.started":"2023-12-15T15:42:17.796783Z","shell.execute_reply":"2023-12-15T15:42:17.801281Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def seedAll(seed):\n random.seed(seed)\n np.random.seed(seed)\n tf.random.set_seed(seed)\n os.environ[\"PYTHONHASHSEED\"]=str(seed)\n\nseedAll(2021)","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:17.803835Z","iopub.execute_input":"2023-12-15T15:42:17.804259Z","iopub.status.idle":"2023-12-15T15:42:17.813704Z","shell.execute_reply.started":"2023-12-15T15:42:17.804222Z","shell.execute_reply":"2023-12-15T15:42:17.812671Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Helpers Army :)","metadata":{}},{"cell_type":"code","source":"def species_id_dis(df, flag, gby=False):\n plt.figure(figsize=(18, 6))\n if not gby:\n sns.countplot(x=\"species_id\", data=df)\n plt.title(f\"Species ID distribution for {flag}\")\n else:\n sns.countplot(x=\"species_id\", hue=\"songtype_id\", data=trtp)\n plt.title(f\"Species ID distribution for {flag} grouped by song_type\") \n \n plt.show()\n pass\n\ndef pie_st(df, flag, col=\"songtype_id\"):\n plt.figure(figsize=(8, 8))\n wegdes, texts, autotexts = plt.pie(df[col].value_counts(), \n startangle=45, \n wedgeprops={\"linewidth\":1, \"edgecolor\":\"black\"}, \n autopct='%1.f%%', \n shadow=True,\n textprops= dict(color=\"black\"),\n explode=(0.2, 0.2))\n plt.legend(wegdes, df[col].value_counts().index,\n title=\"song type\",\n loc=\"center\",\n bbox_to_anchor=(1, 0, 0, 0))\n plt.setp(autotexts, size=14, weight=\"bold\")\n plt.title(f\"Song types distribution for {flag}\")\n plt.show()\n pass\n\ndef outlier_viz(df, col, flag):\n fig, ax = plt.subplots(1, 2, figsize=(20, 6))\n sns.boxplot(x=col, data=df, orient=\"h\", ax=ax[0])\n sns.distplot(df[col], kde=True, ax=ax[1])\n \n if col==\"clip_duration\":\n fig.suptitle(f\"Distribution of length of audio clips containing any species in {flag}\")\n else:\n fig.suptitle(f\"Distribution of Frequency Ranges containing any species in {flag}\")\n fig.show()\n pass\n\ndef spidwise_viz(df, col, flag):\n plt.figure(figsize=(20, 20))\n sns.boxplot(x=col, y=\"species_id\", data=df, orient=\"h\")\n \n if col==\"clip_duration\":\n plt.title(f\"species_id-wise Distribution of length of audio clips containing any species in {flag} Samples\")\n else:\n plt.title(f\"species_id-wise Distribution of Frequency Ranges containing any species in {flag} Samples\")\n plt.show()\n pass\n\ndef soidwise_viz(col):\n fig, axs = plt.subplots(1, 2, figsize=(22, 8))\n \n if col==\"clip_duration\":\n flag = \"length of audio clips\"\n else:\n flag = \"Frequency Ranges\"\n \n sns.boxplot(y=col, x=\"songtype_id\", data=trtp, ax=axs[0])\n axs[0].set_title(f\"species_id-wise Distribution of {flag} containing any species in True positive Samples\")\n\n sns.boxplot(y=col, x=\"songtype_id\", data=trfp, ax=axs[1])\n axs[1].set_title(f\"species_id-wise Distribution of {flag} containing any species in False positive Samples\")\n\n fig.show()\n pass\n\ndef pivot_data(df, col, gby=\"species_id\"):\n pct_25 = lambda x: np.percentile(x, 25)\n pct_75 = lambda x: np.percentile(x, 75)\n pct_75.__name__ = \"75%\"\n pct_25.__name__ = \"25%\"\n display(df.pivot_table(col, gby, aggfunc=[\"count\", \"min\", pct_25, \"mean\", \"median\", pct_75, \"max\"]).style.background_gradient(cmap=\"plasma\"))\n pass","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:17.815383Z","iopub.execute_input":"2023-12-15T15:42:17.815763Z","iopub.status.idle":"2023-12-15T15:42:17.841804Z","shell.execute_reply.started":"2023-12-15T15:42:17.815730Z","shell.execute_reply":"2023-12-15T15:42:17.840605Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3. Explarotary Data Analysis\n\nJust an information for the readers, as I am an electronics major, I have much more inclination towards Digital signal processing, and I have taken this one as one of my research project, so I will be updating my resources, domain knowledge here or in discussion threads as the competition progresses.","metadata":{}},{"cell_type":"markdown","source":"# 3.1 Train True Positives","metadata":{}},{"cell_type":"code","source":"trtp = pd.read_csv(train_tp)\ntrtp","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:17.846584Z","iopub.execute_input":"2023-12-15T15:42:17.846936Z","iopub.status.idle":"2023-12-15T15:42:17.897471Z","shell.execute_reply.started":"2023-12-15T15:42:17.846896Z","shell.execute_reply":"2023-12-15T15:42:17.896338Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So our target col is `species_id`. Let's have a look at the class counts.","metadata":{}},{"cell_type":"markdown","source":"But what does `songtype_id` represent? ","metadata":{}},{"cell_type":"code","source":"species_id_dis(trtp, \"True Positives\")","metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"execution":{"iopub.status.busy":"2023-12-15T15:42:17.900359Z","iopub.execute_input":"2023-12-15T15:42:17.900719Z","iopub.status.idle":"2023-12-15T15:42:18.213217Z","shell.execute_reply.started":"2023-12-15T15:42:17.900684Z","shell.execute_reply":"2023-12-15T15:42:18.212315Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"species_id_dis(trtp, \"True Positives\", gby=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:18.214476Z","iopub.execute_input":"2023-12-15T15:42:18.214949Z","iopub.status.idle":"2023-12-15T15:42:18.798036Z","shell.execute_reply.started":"2023-12-15T15:42:18.214892Z","shell.execute_reply":"2023-12-15T15:42:18.797272Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pie_st(trtp, \"True Positives\", col=\"songtype_id\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:18.799240Z","iopub.execute_input":"2023-12-15T15:42:18.799714Z","iopub.status.idle":"2023-12-15T15:42:18.900452Z","shell.execute_reply.started":"2023-12-15T15:42:18.799668Z","shell.execute_reply":"2023-12-15T15:42:18.899324Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3.1.1 Target count distribution\n\n- An implication of class imbalance reflects for specied_id 23 with 100 counts.\n- We do not have songtype 4 for any of these classes except species 16, 17 and 23.\n- For species_id 16, there exists only songtype of 4.\n- For species_id 23, there seems to be a perfect balance for both of the song types.\n\nBut again, what is songtype_id? Let's have a look at them.","metadata":{}},{"cell_type":"markdown","source":"# 3.2 Train False Positives","metadata":{}},{"cell_type":"code","source":"trfp = pd.read_csv(train_fp)\ntrfp","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:18.901928Z","iopub.execute_input":"2023-12-15T15:42:18.902574Z","iopub.status.idle":"2023-12-15T15:42:18.944712Z","shell.execute_reply.started":"2023-12-15T15:42:18.902524Z","shell.execute_reply":"2023-12-15T15:42:18.943518Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"species_id_dis(trfp, \"False Positives\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:18.946436Z","iopub.execute_input":"2023-12-15T15:42:18.946908Z","iopub.status.idle":"2023-12-15T15:42:19.224027Z","shell.execute_reply.started":"2023-12-15T15:42:18.946849Z","shell.execute_reply":"2023-12-15T15:42:19.222984Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"species_id_dis(trfp, \"False Positives\", gby=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:19.225497Z","iopub.execute_input":"2023-12-15T15:42:19.225818Z","iopub.status.idle":"2023-12-15T15:42:19.599016Z","shell.execute_reply.started":"2023-12-15T15:42:19.225786Z","shell.execute_reply":"2023-12-15T15:42:19.598012Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pie_st(trfp, \"False Positives\", col=\"songtype_id\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:19.600412Z","iopub.execute_input":"2023-12-15T15:42:19.600726Z","iopub.status.idle":"2023-12-15T15:42:19.688524Z","shell.execute_reply.started":"2023-12-15T15:42:19.600697Z","shell.execute_reply":"2023-12-15T15:42:19.687742Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#

CuriousityN1

\n\n## Let's see how many samples are there, those present in train_tp as well as train_fp?","metadata":{}},{"cell_type":"code","source":"print(\"Total Train audio samples: \", len(os.listdir(train_folder)))\nprint(\"Total Test audio samples: \", len(os.listdir(test_folder)))\nprint(\"Number of samples present both in Train True Positives and Train False Positives: \", len(set(trfp.recording_id.tolist()).intersection(trtp.recording_id.tolist())))\nprint(\"Number of unique audio samples in Train True positives: \", trtp.recording_id.nunique())\nprint(\"Number of unique audio samples in Train False positives: \", trfp.recording_id.nunique())","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:19.689594Z","iopub.execute_input":"2023-12-15T15:42:19.689993Z","iopub.status.idle":"2023-12-15T15:42:25.871924Z","shell.execute_reply.started":"2023-12-15T15:42:19.689962Z","shell.execute_reply":"2023-12-15T15:42:25.871069Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3.2.1 Clip Duration Analysis\n\n## Species ID-wise","metadata":{}},{"cell_type":"code","source":"trtp[\"clip_duration\"] = trtp[\"t_max\"] - trtp[\"t_min\"]\ntrfp[\"clip_duration\"] = trfp[\"t_max\"] - trfp[\"t_min\"]","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:25.873424Z","iopub.execute_input":"2023-12-15T15:42:25.873778Z","iopub.status.idle":"2023-12-15T15:42:25.882673Z","shell.execute_reply.started":"2023-12-15T15:42:25.873741Z","shell.execute_reply":"2023-12-15T15:42:25.881232Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"outlier_viz(trtp, \"clip_duration\", \"True Positives\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:25.884664Z","iopub.execute_input":"2023-12-15T15:42:25.885093Z","iopub.status.idle":"2023-12-15T15:42:26.209354Z","shell.execute_reply.started":"2023-12-15T15:42:25.885056Z","shell.execute_reply":"2023-12-15T15:42:26.208252Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spidwise_viz(trtp, \"clip_duration\", \"True Positives\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:26.210808Z","iopub.execute_input":"2023-12-15T15:42:26.211121Z","iopub.status.idle":"2023-12-15T15:42:26.798635Z","shell.execute_reply.started":"2023-12-15T15:42:26.211092Z","shell.execute_reply":"2023-12-15T15:42:26.797707Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pivot_data(trtp, \"clip_duration\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:26.800133Z","iopub.execute_input":"2023-12-15T15:42:26.800487Z","iopub.status.idle":"2023-12-15T15:42:27.066566Z","shell.execute_reply.started":"2023-12-15T15:42:26.800452Z","shell.execute_reply":"2023-12-15T15:42:27.065336Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"outlier_viz(trfp, \"clip_duration\", \"False Positives\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:27.068514Z","iopub.execute_input":"2023-12-15T15:42:27.068854Z","iopub.status.idle":"2023-12-15T15:42:27.434245Z","shell.execute_reply.started":"2023-12-15T15:42:27.068823Z","shell.execute_reply":"2023-12-15T15:42:27.433352Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spidwise_viz(trfp, \"clip_duration\", \"False Positives\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:27.435664Z","iopub.execute_input":"2023-12-15T15:42:27.435984Z","iopub.status.idle":"2023-12-15T15:42:28.034120Z","shell.execute_reply.started":"2023-12-15T15:42:27.435952Z","shell.execute_reply":"2023-12-15T15:42:28.033091Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pivot_data(trfp, \"clip_duration\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:28.035721Z","iopub.execute_input":"2023-12-15T15:42:28.036069Z","iopub.status.idle":"2023-12-15T15:42:28.164588Z","shell.execute_reply.started":"2023-12-15T15:42:28.036034Z","shell.execute_reply":"2023-12-15T15:42:28.163505Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3.2.2 Frequency Range Analysis","metadata":{}},{"cell_type":"code","source":"trtp[\"freq_range\"] = trtp[\"f_max\"] - trtp[\"f_min\"]\ntrfp[\"freq_range\"] = trfp[\"f_max\"] - trfp[\"f_min\"]","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:28.166140Z","iopub.execute_input":"2023-12-15T15:42:28.166767Z","iopub.status.idle":"2023-12-15T15:42:28.176138Z","shell.execute_reply.started":"2023-12-15T15:42:28.166719Z","shell.execute_reply":"2023-12-15T15:42:28.175267Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"outlier_viz(trtp, \"freq_range\", \"True Positives\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:28.177777Z","iopub.execute_input":"2023-12-15T15:42:28.178118Z","iopub.status.idle":"2023-12-15T15:42:28.494187Z","shell.execute_reply.started":"2023-12-15T15:42:28.178078Z","shell.execute_reply":"2023-12-15T15:42:28.493316Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spidwise_viz(trtp, \"freq_range\", \"True Positives\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:28.495680Z","iopub.execute_input":"2023-12-15T15:42:28.496319Z","iopub.status.idle":"2023-12-15T15:42:29.377253Z","shell.execute_reply.started":"2023-12-15T15:42:28.496270Z","shell.execute_reply":"2023-12-15T15:42:29.376079Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pivot_data(trtp, \"freq_range\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:29.378928Z","iopub.execute_input":"2023-12-15T15:42:29.379573Z","iopub.status.idle":"2023-12-15T15:42:29.511895Z","shell.execute_reply.started":"2023-12-15T15:42:29.379520Z","shell.execute_reply":"2023-12-15T15:42:29.510820Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"outlier_viz(trfp, \"freq_range\", \"False Positives\")","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:29.513392Z","iopub.execute_input":"2023-12-15T15:42:29.513817Z","iopub.status.idle":"2023-12-15T15:42:29.863446Z","shell.execute_reply.started":"2023-12-15T15:42:29.513772Z","shell.execute_reply":"2023-12-15T15:42:29.862325Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spidwise_viz(trfp, \"freq_range\", \"False Positives\")","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:29.865133Z","iopub.execute_input":"2023-12-15T15:42:29.865859Z","iopub.status.idle":"2023-12-15T15:42:30.451361Z","shell.execute_reply.started":"2023-12-15T15:42:29.865812Z","shell.execute_reply":"2023-12-15T15:42:30.450205Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pivot_data(trfp, \"freq_range\")","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:30.453307Z","iopub.execute_input":"2023-12-15T15:42:30.453784Z","iopub.status.idle":"2023-12-15T15:42:30.584325Z","shell.execute_reply.started":"2023-12-15T15:42:30.453737Z","shell.execute_reply":"2023-12-15T15:42:30.583055Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Songtype_id-wise","metadata":{}},{"cell_type":"code","source":"soidwise_viz(\"clip_duration\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:30.586335Z","iopub.execute_input":"2023-12-15T15:42:30.586802Z","iopub.status.idle":"2023-12-15T15:42:30.901344Z","shell.execute_reply.started":"2023-12-15T15:42:30.586754Z","shell.execute_reply":"2023-12-15T15:42:30.900207Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pivot_data(trtp, col=\"clip_duration\", gby=\"songtype_id\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:30.903291Z","iopub.execute_input":"2023-12-15T15:42:30.903756Z","iopub.status.idle":"2023-12-15T15:42:31.004086Z","shell.execute_reply.started":"2023-12-15T15:42:30.903707Z","shell.execute_reply":"2023-12-15T15:42:31.002949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pivot_data(trfp, col=\"clip_duration\", gby=\"songtype_id\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:31.005569Z","iopub.execute_input":"2023-12-15T15:42:31.005880Z","iopub.status.idle":"2023-12-15T15:42:31.107556Z","shell.execute_reply.started":"2023-12-15T15:42:31.005850Z","shell.execute_reply":"2023-12-15T15:42:31.106603Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"soidwise_viz(\"freq_range\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:31.109125Z","iopub.execute_input":"2023-12-15T15:42:31.109491Z","iopub.status.idle":"2023-12-15T15:42:31.407464Z","shell.execute_reply.started":"2023-12-15T15:42:31.109453Z","shell.execute_reply":"2023-12-15T15:42:31.406318Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pivot_data(trtp, col=\"freq_range\", gby=\"songtype_id\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:31.415480Z","iopub.execute_input":"2023-12-15T15:42:31.415873Z","iopub.status.idle":"2023-12-15T15:42:31.516807Z","shell.execute_reply.started":"2023-12-15T15:42:31.415837Z","shell.execute_reply":"2023-12-15T15:42:31.515821Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pivot_data(trfp, col=\"freq_range\", gby=\"songtype_id\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:31.518037Z","iopub.execute_input":"2023-12-15T15:42:31.518364Z","iopub.status.idle":"2023-12-15T15:42:31.620274Z","shell.execute_reply.started":"2023-12-15T15:42:31.518334Z","shell.execute_reply":"2023-12-15T15:42:31.619173Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3.3 Analysing Audio data with [Librosa](https://librosa.org/doc/latest/index.html)","metadata":{}},{"cell_type":"markdown","source":"# 3.3.1 Load and Examine the audio sample","metadata":{}},{"cell_type":"code","source":"# choose a sample from train and test\ntr = os.path.join(train_folder, os.listdir(train_folder)[np.random.randint(0, len(os.listdir(train_folder)))])\nts = os.path.join(test_folder, os.listdir(test_folder)[np.random.randint(0, len(os.listdir(test_folder)))])\n\n# load the np array and the samping rate\ntrx, trsr = librosa.load(tr)\ntsx, tssr = librosa.load(ts)\nrecId_train = (tr.split(\"/\")[-1]).split(\".\")[0]\nrecId_test = (ts.split(\"/\")[-1]).split(\".\")[0]\n\nprint(\"=\"*10, \"Training Sample\", \"=\"*10)\ndisplay(trfp[trfp[\"recording_id\"]==recId_train] if recId_train in trfp[\"recording_id\"].tolist() else trtp[trtp[\"recording_id\"]==recId_train])\nprint(\"=\"*10, \"Test Sample\", \"=\"*10)\nprint(\"Test Data: \", recId_test)","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:31.621740Z","iopub.execute_input":"2023-12-15T15:42:31.622074Z","iopub.status.idle":"2023-12-15T15:42:38.018186Z","shell.execute_reply.started":"2023-12-15T15:42:31.622043Z","shell.execute_reply":"2023-12-15T15:42:38.016843Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So in our picked sample(recId=efcc3bd16), we have one single species i.e species_20. It's heard between 49.232s to 52.672s and the min and max freq components are 2343.75hz and 5718.75Hz respectively. Let's explore the priliminary features.","metadata":{}},{"cell_type":"code","source":"# print the shape and the sampling rate\nprint(\"=====Train sample=======\")\nprint(trx.shape, trsr)\nprint(\"=====Test sample=======\")\nprint(tsx.shape, tssr)","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:38.019698Z","iopub.execute_input":"2023-12-15T15:42:38.020115Z","iopub.status.idle":"2023-12-15T15:42:38.026082Z","shell.execute_reply.started":"2023-12-15T15:42:38.020078Z","shell.execute_reply":"2023-12-15T15:42:38.025096Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So, yes, each data sample is an 1-D Vector with shape (1323000, ) and sampling rate 22050 (? how does librosa do that?). Oh and if you haven't noticed yet, the audio files are just named as `recording_id.flac`. So, let's have a walk around.","metadata":{}},{"cell_type":"markdown","source":"# 3.3.2 Play the Audio","metadata":{}},{"cell_type":"markdown","source":"### Train sample","metadata":{}},{"cell_type":"code","source":"ipD.Audio(tr)","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:38.027676Z","iopub.execute_input":"2023-12-15T15:42:38.027992Z","iopub.status.idle":"2023-12-15T15:42:38.109294Z","shell.execute_reply.started":"2023-12-15T15:42:38.027959Z","shell.execute_reply":"2023-12-15T15:42:38.107326Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Test sample","metadata":{}},{"cell_type":"code","source":"ipD.Audio(ts)","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:38.110921Z","iopub.execute_input":"2023-12-15T15:42:38.111465Z","iopub.status.idle":"2023-12-15T15:42:38.196683Z","shell.execute_reply.started":"2023-12-15T15:42:38.111388Z","shell.execute_reply":"2023-12-15T15:42:38.195114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3.3.3 Waveplot of the audio samples","metadata":{}},{"cell_type":"markdown","source":"### Train sample","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(14, 5))\nLD.waveplot(trx, sr=trsr)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:38.198800Z","iopub.execute_input":"2023-12-15T15:42:38.199342Z","iopub.status.idle":"2023-12-15T15:42:38.504971Z","shell.execute_reply.started":"2023-12-15T15:42:38.199285Z","shell.execute_reply":"2023-12-15T15:42:38.504058Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Test Sample","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(14, 5))\nLD.waveplot(tsx, sr=tssr)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:38.506277Z","iopub.execute_input":"2023-12-15T15:42:38.506579Z","iopub.status.idle":"2023-12-15T15:42:38.854646Z","shell.execute_reply.started":"2023-12-15T15:42:38.506549Z","shell.execute_reply":"2023-12-15T15:42:38.853400Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3.3.4 Short Time Fourier Transform(STFT)","metadata":{}},{"cell_type":"markdown","source":"But what is a Short time fourier transform? What happened to General DFT? Let's understand. \n\nA signal may contain one or more frequency components. But the signal representation doesn't tell about the frequency components present in a signal. To do so, we need **Fourier Transform**. It tells about the frequency components present in a signal. \n\nThen what's the problem? Why do we even need STFT? \n\nBecause there remains a fundamental trade-off between time and frequency. Here is a quick explanatory [video](https://www.youtube.com/watch?v=g1_wcbGUcDY). The time represenation obfuscates frequency, and so does frequency representation obfuscates frequency. In no single representation, we have a clear picture of both of them. Here comes the idea of Short time fourier transform.\n\n**Short-time Fourier transform (STFT)** is a sequence of Fourier transforms of a windowed signal. STFT provides the time-localized frequency information for situations in which frequency components of a signal vary over time, whereas the standard Fourier transform provides the frequency information averaged over the entire signal time interval.\n\nIn simpler words, we take **fixed-length Windows of signals** from the original signal and apply fourier transform to each window and then take the sum over all the windows. Here I am attachingn a few resources to follow:\n\n1. YouTube Video: https://www.youtube.com/watch?v=g1_wcbGUcDY\n2. Science Direct: https://www.sciencedirect.com/topics/engineering/short-time-fourier-transform\n\nStill, there remains ambiguities over selecting a window length, shape of the window, window filters, procssings, etc. We willl dig deep as the competition progresses.\n\nAnd this stft is best visuallized by spectogram plot. But what is a spectogram? It's a spectrum of frequency of a signal. Okay, let's worry about hwo to obtain it and what to do iwth this feature?","metadata":{}},{"cell_type":"markdown","source":"### Train sample","metadata":{}},{"cell_type":"code","source":"TRX = librosa.stft(trx)\nprint(\"Shape of the stft: \", TRX.shape)\n# convert into db\nTRXdb = librosa.amplitude_to_db(abs(TRX))\n\nplt.figure(figsize=(14, 5))\nlibrosa.display.specshow(TRXdb, sr=trsr, x_axis='time', y_axis='hz')\nplt.colorbar()\nplt.figure(figsize=(14, 5))\nlibrosa.display.specshow(TRXdb, sr=trsr, x_axis='time', y_axis='log')\nplt.title(\"In log scale\")\nplt.colorbar()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:38.856140Z","iopub.execute_input":"2023-12-15T15:42:38.856557Z","iopub.status.idle":"2023-12-15T15:42:42.799130Z","shell.execute_reply.started":"2023-12-15T15:42:38.856526Z","shell.execute_reply":"2023-12-15T15:42:42.798205Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Test Sample","metadata":{}},{"cell_type":"code","source":"# convert into fourier transform\nTSX = librosa.stft(tsx)\nprint(\"Shape of the stft: \", TSX.shape)\n\n# convert into bd\nTSXdb = librosa.amplitude_to_db(abs(TSX))\nplt.figure(figsize=(14, 5))\nlibrosa.display.specshow(TSXdb, sr=tssr, x_axis='time', y_axis='hz')\nplt.colorbar()\nplt.figure(figsize=(14, 5))\nlibrosa.display.specshow(TSXdb, sr=tssr, x_axis='time', y_axis='log')\nplt.colorbar()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:42.800392Z","iopub.execute_input":"2023-12-15T15:42:42.800674Z","iopub.status.idle":"2023-12-15T15:42:46.552510Z","shell.execute_reply.started":"2023-12-15T15:42:42.800647Z","shell.execute_reply":"2023-12-15T15:42:46.551457Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3.3.5 Spectral Centroids","metadata":{}},{"cell_type":"markdown","source":"Let's understand what's a Spectral Centroid. As the name sugests, it's the **centroid of Spectral components**. Well,that's obvious from the name, what else? It indicates where the ”centre of mass” for a sound is located and is calculated as the weighted mean of the frequencies present in the sound. Or in simple words: It gives the measure of the brightness of a sound. The individual centroid of a spectral frame is defined as the average frequency weighted by amplitudes, divided by the sum of the amplitudes.","metadata":{}},{"cell_type":"markdown","source":"### Train sample","metadata":{}},{"cell_type":"code","source":"spectral_centroids = librosa.feature.spectral_centroid(trx, sr=trsr)[0]\nprint(\"Shape of the spectral centroids: \", spectral_centroids.shape)\n\n# extract the time and frame indices\nframes = range(len(spectral_centroids))\nt = librosa.frames_to_time(frames)\n\nplt.figure(figsize=(10, 4))\nLD.waveplot(trx, sr=trsr, alpha=0.4)\nplt.title(\"Spectral Centroids for Train sample\")\nplt.show()\n\nplt.figure(figsize=(10, 4))\nLD.waveplot(trx, sr=trsr, alpha=0.4)\nplt.plot(t, ss.fit_transform(spectral_centroids.reshape(-1, 1)))\nplt.plot(t, mm.fit_transform(spectral_centroids.reshape(-1, 1)))\nplt.legend([\"Audio Signal\", \"sc_ss\", \"sc_mm\"][::-1])\nplt.title(\"Normalized Spectral Centroid Visualization for Train sample\")\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:46.554374Z","iopub.execute_input":"2023-12-15T15:42:46.554934Z","iopub.status.idle":"2023-12-15T15:42:47.212022Z","shell.execute_reply.started":"2023-12-15T15:42:46.554886Z","shell.execute_reply":"2023-12-15T15:42:47.210925Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Test sample","metadata":{}},{"cell_type":"code","source":"spectral_centroids = librosa.feature.spectral_centroid(tsx, sr=tssr)[0]\nprint(\"Shape of the spectral centroids: \", spectral_centroids.shape)\n\n# extract the time and frame indices\nframes = range(len(spectral_centroids))\nt = librosa.frames_to_time(frames)\n\nplt.figure(figsize=(10, 4))\nLD.waveplot(tsx, sr=tssr, alpha=0.4)\nplt.title(\"Spectral Centroids for Test sample\")\nplt.show()\n\nplt.figure(figsize=(10, 4))\nLD.waveplot(tsx, sr=tssr, alpha=0.4)\nplt.plot(t, ss.fit_transform(spectral_centroids.reshape(-1, 1)))\nplt.plot(t, mm.fit_transform(spectral_centroids.reshape(-1, 1)))\nplt.legend([\"Audio Signal\", \"sc_ss\", \"sc_mm\"][::-1])\nplt.title(\"Normalized Spectral Centroid Visualization for Test sample\")\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-12-15T15:42:47.213520Z","iopub.execute_input":"2023-12-15T15:42:47.213812Z","iopub.status.idle":"2023-12-15T15:42:47.923406Z","shell.execute_reply.started":"2023-12-15T15:42:47.213784Z","shell.execute_reply":"2023-12-15T15:42:47.920766Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3.3.6 Spectral Roll-off","metadata":{}},{"cell_type":"code","source":"spectral_rolloff = librosa.feature.spectral_rolloff(trx+0.01, sr=trsr)\nframes = range(len(spectral_rolloff))\nt = librosa.frames_to_time(frames)\n\nplt.figure(figsize=(12, 4))\nLD.waveplot(trx, sr=trsr, alpha=0.4)\nplt.plot(t, ss.fit_transform(spectral_rolloff), color='r')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:47.924839Z","iopub.execute_input":"2023-12-15T15:42:47.925199Z","iopub.status.idle":"2023-12-15T15:42:51.034364Z","shell.execute_reply.started":"2023-12-15T15:42:47.925153Z","shell.execute_reply":"2023-12-15T15:42:51.033294Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spectral_rolloff = librosa.feature.spectral_rolloff(tsx, sr=tssr)\nframes = range(len(spectral_rolloff))\nt = librosa.frames_to_time(frames)\n\nplt.figure(figsize=(10, 4))\nLD.waveplot(tsx, sr=tssr, alpha=0.4)\nplt.plot(t, ss.fit_transform(spectral_rolloff), color=\"r\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:51.035569Z","iopub.execute_input":"2023-12-15T15:42:51.035883Z","iopub.status.idle":"2023-12-15T15:42:54.094097Z","shell.execute_reply.started":"2023-12-15T15:42:51.035854Z","shell.execute_reply":"2023-12-15T15:42:54.093296Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3.3.7 Mel-Frequency Cepstral Coefficients(MFCCs)","metadata":{}},{"cell_type":"markdown","source":"Everyone is using MFCC, huh? This must be super cool. Well, let's dive deep together. `You want to conquer something, break it down to pieces`: Don't who said this, or probably I quoted it randomly, but let's follow the traditional approach of understanding a new concept.\n\n**MFCC**: Mel Frequency Cepstral Coefficients\n\n# Mel\n\n- **Mel scale** is a scale that relates **the perceived frequency of a tone** to **the actual measured frequency**. \n- It scales the frequency in order to match more closely what the human ear can hear (humans are better at identifying small changes in speech at lower frequencies). This scale has been derived from sets of experiments on human subjects. \n\nLet me give you an intuitive explanation of what the mel scale captures:\n\nThe range of human hearing is 20Hz to 20kHz. Imagine a tune at 300 Hz. This would sound something like the standard dialer tone of a land-line phone. Now imagine a tune at 400 Hz (a little higher pitched dialer tone). Now compare the distance between these two howsoever this may be perceived by your brain. \n\nNow imagine a 900 Hz signal (similar to a microphone feedback sound) and a 1kHz sound. The perceived distance between these two sounds may seem greater than the first two although the actual difference is the same (100Hz). \n\nThe mel scale tries to capture such differences. A frequency measured in Hertz (f) can be converted to the Mel scale using the following formula :\n$$Mel(f) = 2595\\log(1 + \\frac{f}{700})$$\n\n# Frequency\n\nWell, you know that I know that you know I know :) But, why Frequency domain analysis? Explained [here](https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/201129).\n\n# Cepstral\n\nOn a lighter note, `\"speC\"[::-1] + \"tral\"` ;) Well, jokes apart, or was it a joke? Nope xD. Here's why and how? \nLet's understand what is **cepstrum?** For a very basic understanding, cepstrum is the **information of rate of change in spectral bands**. In the conventional analysis of time signals, any periodic component (for eg, echoes) shows up as sharp peaks in the corresponding frequency spectrum (ie, Fourier spectrum(obtained by fourier transform)).\n\nOn taking the log of the magnitude of this Fourier spectrum, and then again taking the spectrum of this log by a cosine transformation, we observe a peak wherever there is a periodic element in the original time signal. **TL;DR**,`cosine_transform(log(mag(fourier_transformation)))`.\n\nSince we apply a transform on the frequency spectrum itself, the resulting spectrum is neither in the frequency domain nor in the time domain and hence [Bogert et al.](https://www.fceia.unr.edu.ar/prodivoz/Oppenheim_Schafer_2004.pdf) decided to call it the ***quefrency domain***. And this spectrum of the log of the spectrum of the time signal was named **cepstrum**.\n\n# Coefficients\n\nCoefficients are nothing but what makes up the cepstrum.\n\nBeing saig, Any sound generated by humans is determined by the shape of their vocal tract (including tongue, teeth, etc). If this shape can be determined correctly, any sound produced can be accurately represented. The envelope of the **time power spectrum of the speech signal** is representative of the vocal tract and MFCCs accurately represents this envelope. Here is a diagram explaining the whole process:\n![cepstralCoeff](https://miro.medium.com/max/788/1*dWnjn5LLS0j8St53ACwqSg.jpeg)\n\nThe Cepstral coeffs mentioned in the block diagram are nothing but the MFCCs.","metadata":{}},{"cell_type":"markdown","source":"But why **power spectrum**? I searched and found this.\n\nWhat Andrew Ng said:\n> phonemes, which are the smallest components of sound, didn’t matter. To a certain extent, in fact, what matters is that voice is mostly a (quasi) periodic signal if time intervals are small enough. \n\nThis leads to the idea of ignoring the phase of the signal and only using its power spectrum as a source of information. The fact that sound can be reconstructed from its power spectrum (with the Griffin-Lim algorithm or a neural vocoder, for example) proves that this is the case. I will come back to the discussion later.","metadata":{}},{"cell_type":"markdown","source":"### Train sample","metadata":{}},{"cell_type":"code","source":"mfccs = librosa.feature.mfcc(trx, sr=trsr)\nprint(mfccs.shape)\n#Displaying the MFCCs:\nplt.figure(figsize=(15, 7))\nLD.specshow(mfccs, sr=trsr, x_axis='time')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:54.095202Z","iopub.execute_input":"2023-12-15T15:42:54.095613Z","iopub.status.idle":"2023-12-15T15:42:54.423667Z","shell.execute_reply.started":"2023-12-15T15:42:54.095582Z","shell.execute_reply":"2023-12-15T15:42:54.422718Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Test Sample","metadata":{}},{"cell_type":"code","source":"mfccs = librosa.feature.mfcc(tsx, sr=tssr)\nprint(mfccs.shape)\n#Displaying the MFCCs:\nplt.figure(figsize=(15, 7))\nLD.specshow(mfccs, sr=tssr, x_axis='time')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:54.424924Z","iopub.execute_input":"2023-12-15T15:42:54.425257Z","iopub.status.idle":"2023-12-15T15:42:54.713800Z","shell.execute_reply.started":"2023-12-15T15:42:54.425224Z","shell.execute_reply":"2023-12-15T15:42:54.712790Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Mel Spectrogram","metadata":{}},{"cell_type":"code","source":"melspec = librosa.feature.melspectrogram(trx, sr=trsr)\nprint(melspec.shape)\nplt.figure(figsize=(10, 4))\nlibrosa.display.specshow(librosa.power_to_db(melspec, ref=np.max),\n y_axis='mel',\n x_axis='time')\nplt.colorbar(format='%+2.0f dB')\nplt.title('Mel spectrogram')\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:54.717353Z","iopub.execute_input":"2023-12-15T15:42:54.717708Z","iopub.status.idle":"2023-12-15T15:42:55.328739Z","shell.execute_reply.started":"2023-12-15T15:42:54.717671Z","shell.execute_reply":"2023-12-15T15:42:55.327545Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Chroma feature","metadata":{}},{"cell_type":"markdown","source":"### Train sample","metadata":{}},{"cell_type":"code","source":"hop_length=12\nchromagram = librosa.feature.chroma_stft(trx, sr=trsr, hop_length=hop_length)\nplt.figure(figsize=(15, 5))\nLD.specshow(chromagram, x_axis='time', y_axis='chroma', hop_length=hop_length, cmap='coolwarm')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:42:55.330377Z","iopub.execute_input":"2023-12-15T15:42:55.330764Z","iopub.status.idle":"2023-12-15T15:43:06.815832Z","shell.execute_reply.started":"2023-12-15T15:42:55.330719Z","shell.execute_reply":"2023-12-15T15:43:06.814888Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Test Sample","metadata":{}},{"cell_type":"code","source":"chromagram = librosa.feature.chroma_stft(tsx, sr=tssr, hop_length=hop_length)\nplt.figure(figsize=(15, 5))\nLD.specshow(chromagram, x_axis='time', y_axis='chroma', hop_length=hop_length, cmap='coolwarm')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:43:06.817290Z","iopub.execute_input":"2023-12-15T15:43:06.817610Z","iopub.status.idle":"2023-12-15T15:43:17.413903Z","shell.execute_reply.started":"2023-12-15T15:43:06.817578Z","shell.execute_reply":"2023-12-15T15:43:17.412781Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3.4 Sample Submission","metadata":{}},{"cell_type":"code","source":"samSub = pd.read_csv(sample_submission)\nsamSub","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:43:17.415614Z","iopub.execute_input":"2023-12-15T15:43:17.416073Z","iopub.status.idle":"2023-12-15T15:43:17.470386Z","shell.execute_reply.started":"2023-12-15T15:43:17.416026Z","shell.execute_reply":"2023-12-15T15:43:17.469296Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 4. Explore TFRecords","metadata":{}},{"cell_type":"code","source":"train_tfrec = \"../input/rfcx-species-audio-detection/tfrecords/train\"\ntest_tfrec = \"../input/rfcx-species-audio-detection/tfrecords/test\"\n\ntrain_tfrecs = sorted(tf.io.gfile.glob(train_tfrec + \"/*.tfrec\"))\ntest_tfrecs = sorted(tf.io.gfile.glob(test_tfrec + \"/*.tfrec\"))\n\nprint(\"Number of train tfrecords: \", len(train_tfrecs))\nprint(\"Number of test tfrecords: \", len(test_tfrecs))","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:43:17.472073Z","iopub.execute_input":"2023-12-15T15:43:17.472543Z","iopub.status.idle":"2023-12-15T15:43:18.010988Z","shell.execute_reply.started":"2023-12-15T15:43:17.472504Z","shell.execute_reply":"2023-12-15T15:43:18.010099Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Read a tfrec sample to inspect. Following is the syntax for reading a tfrec data in tf. \n`tf.data.TFRecordDataset([filename])`","metadata":{}},{"cell_type":"code","source":"sample_proto_train_tfrec = tf.data.TFRecordDataset([train_tfrecs[0]])\nsample_proto_train_tfrec","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:43:18.014797Z","iopub.execute_input":"2023-12-15T15:43:18.015141Z","iopub.status.idle":"2023-12-15T15:43:18.132781Z","shell.execute_reply.started":"2023-12-15T15:43:18.015110Z","shell.execute_reply":"2023-12-15T15:43:18.131896Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now, let's go back to the data description and see what they have mentioned about the tfrecords.\n\n> competition data in the TFRecord format, which includes **recording_id**, **audio_wav** (encoded in 16-bit PCM format), and **label_info** (for train only), which provides a `,` -delimited string of the columns (except recording_id), where multiple labels for a recording_id are `;` -delimited.","metadata":{}},{"cell_type":"markdown","source":"As we can see there are 148 samples there in each tfrecords , ofc the last samples named `../input/rfcx-species-audio-detection/tfrecords/train/31-139.tfrec` and for test its 63 and `../input/rfcx-species-audio-detection/tfrecords/test/31-39.tfrec` hence 39 respectively. Okay, let's just have a look below then we will proceed to decode the tfrecs.","metadata":{}},{"cell_type":"code","source":"print(\"Number of samples in one single record: \", sample_proto_train_tfrec.reduce(np.int64(), lambda x, _: x+1).numpy())","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:43:18.134483Z","iopub.execute_input":"2023-12-15T15:43:18.134896Z","iopub.status.idle":"2023-12-15T15:43:28.745764Z","shell.execute_reply.started":"2023-12-15T15:43:18.134861Z","shell.execute_reply":"2023-12-15T15:43:28.744970Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Okay, now that we know our tfrecords contain 3 things (for training samples and 2 for testing), let's create a feature description for each sample inside our tfrecord.\n\n***For more info on tfrecords, head over to [tensorflow documentation.](https://www.tensorflow.org/tutorials/load_data/tfrecord#tfrecord_files_using_tfdata)***\n\nOr mention in the comment, I will write a detailed explanation if required.","metadata":{}},{"cell_type":"markdown","source":"# 4.1 Parse TFRecords","metadata":{}},{"cell_type":"code","source":"CUT_TIME = 10 # cutting window in seconds\nSAMPLE_TIME = 6\nSR = 48000\nFMAX = 24000\nFMIN = 40\n# feature description for the tfrecords\n# this will parsed as arguments into tf.io.parse_single_example\nfeature_description = {\n 'recording_id': tf.io.FixedLenFeature([], tf.string, default_value=''),\n 'audio_wav': tf.io.FixedLenFeature([], tf.string, default_value=''),\n 'label_info': tf.io.FixedLenFeature([], tf.string, default_value=''),\n}\nparse_dtype = {\n 'audio_wav': tf.float32,\n 'recording_id': tf.string,\n 'species_id': tf.int32,\n 'songtype_id': tf.int32,\n 't_min': tf.float32,\n 'f_min': tf.float32,\n 't_max': tf.float32,\n 'f_max':tf.float32,\n 'is_tp': tf.int32\n}\n\n# define a fun to read the encoded tfrec\n@tf.function\ndef _parse_fun(sample):\n sample = tf.io.parse_single_example(sample, feature_description) # this returns a dicionary of the features for a single tfrec\n audio, _ = tf.audio.decode_wav(sample[\"audio_wav\"], desired_channels=1)\n label_info = tf.strings.split(sample[\"label_info\"], \";\")\n labels = tf.strings.regex_replace(label_info, '\"', '')\n \n @tf.function\n def _cut_audio(label):\n items = tf.strings.split(label, sep=',')\n spid = tf.squeeze(tf.strings.to_number(items[0], tf.int32))\n soid = tf.squeeze(tf.strings.to_number(items[1], tf.int32))\n tmin = tf.squeeze(tf.strings.to_number(items[2]))\n fmin = tf.squeeze(tf.strings.to_number(items[3]))\n tmax = tf.squeeze(tf.strings.to_number(items[4]))\n fmax = tf.squeeze(tf.strings.to_number(items[5]))\n tp = tf.squeeze(tf.strings.to_number(items[6], tf.int32))\n\n tmax_s = tmax * tf.cast(SR, tf.float32)\n tmin_s = tmin * tf.cast(SR, tf.float32)\n cut_s = tf.cast(CUT_TIME * SR, tf.float32)\n all_s = tf.cast(60 * SR, tf.float32)\n tsize_s = tmax_s - tmin_s\n cut_min = tf.cast(\n tf.maximum(0.0, \n tf.minimum(tmin_s - (cut_s - tsize_s) / 2,\n tf.minimum(tmax_s + (cut_s - tsize_s) / 2,\n all_s) - cut_s)\n ), tf.int32\n )\n cut_max = cut_min + CUT_TIME * SR\n \n _sample = {\n 'audio_wav': tf.reshape(audio[cut_min:cut_max], [CUT_TIME*SR]),\n 'recording_id': sample['recording_id'],\n 'species_id': spid,\n 'songtype_id': soid,\n 't_min': tmin - tf.cast(cut_min, tf.float32)/tf.cast(SR, tf.float32),\n 'f_min': fmin,\n 't_max': tmax - tf.cast(cut_min, tf.float32)/tf.cast(SR, tf.float32),\n 'f_max': fmax,\n 'is_tp': tp\n }\n return _sample\n \n samples = tf.map_fn(_cut_audio, labels, dtype=parse_dtype)\n return samples\n pass","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:43:28.747057Z","iopub.execute_input":"2023-12-15T15:43:28.747580Z","iopub.status.idle":"2023-12-15T15:43:28.769973Z","shell.execute_reply.started":"2023-12-15T15:43:28.747545Z","shell.execute_reply":"2023-12-15T15:43:28.769195Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"parsed_tfrecs_sample = sample_proto_train_tfrec.map(_parse_fun).unbatch()","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:43:28.771017Z","iopub.execute_input":"2023-12-15T15:43:28.771348Z","iopub.status.idle":"2023-12-15T15:43:29.381660Z","shell.execute_reply.started":"2023-12-15T15:43:28.771316Z","shell.execute_reply":"2023-12-15T15:43:29.380758Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we have samples of our tfrec file decoded and saved in parsed_tfrecs_sample in a serialized manner, which can be iterated over using next iter as following: ","metadata":{}},{"cell_type":"code","source":"sample = next(iter(parsed_tfrecs_sample))\n\nfor key, val in sample.items():\n print(key, \":\", val)","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:43:29.383360Z","iopub.execute_input":"2023-12-15T15:43:29.384124Z","iopub.status.idle":"2023-12-15T15:43:29.657816Z","shell.execute_reply.started":"2023-12-15T15:43:29.384074Z","shell.execute_reply":"2023-12-15T15:43:29.656836Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 4.2 Cut down audio to smaller clips","metadata":{}},{"cell_type":"code","source":"@tf.function\ndef cut_audio(sample, istrain=True):\n # random cutting for train samples\n if istrain:\n cut_min = tf.random.uniform([],\n maxval=(CUT_TIME-SAMPLE_TIME) * SR,\n dtype=tf.int32)\n else:\n # center cropping for validation data\n cut_min = (CUT_TIME - SAMPLE_TIME) * SR//2\n cut_max = cut_min + SAMPLE_TIME * SR\n cutaudio = tf.reshape(\n sample[\"audio_wav\"][cut_min:cut_max], [SAMPLE_TIME * SR]\n )\n\n result = {}\n result.update(sample)\n result[\"audio_wav\"] = cutaudio\n result[\"t_min\"] = tf.maximum(0.0, sample[\"t_min\"] - tf.cast(cut_min, tf.float32)/SR)\n result[\"t_max\"] = tf.maximum(0.0, sample[\"t_max\"] - tf.cast(cut_min, tf.float32)/SR)\n return result\n pass","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:43:29.661502Z","iopub.execute_input":"2023-12-15T15:43:29.661827Z","iopub.status.idle":"2023-12-15T15:43:29.672411Z","shell.execute_reply.started":"2023-12-15T15:43:29.661797Z","shell.execute_reply":"2023-12-15T15:43:29.671512Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 4.3 Create log mel-spectogram features","metadata":{}},{"cell_type":"code","source":"@tf.function\ndef waveToSpec(sample):\n mel_power = 2\n stfts = tf.signal.stft(sample[\"audio_wav\"],\n frame_length=2048,\n frame_step=512,\n fft_length=2048)\n spectograms = tf.abs(stfts) ** mel_power\n\n # convert into mel scale\n mel_weight = tf.signal.linear_to_mel_weight_matrix(\n num_mel_bins=224, # or can be said as the no of MFCCs, though theoretically, ideal value should be in 30-50 range, let's try with 224 for image size\n num_spectrogram_bins=stfts.shape[-1],\n sample_rate=SR,\n lower_edge_hertz=FMIN,\n upper_edge_hertz=FMAX\n )\n mel_spectrograms = tf.tensordot(\n spectograms, mel_weight, 1\n )\n mel_spectrograms.set_shape(spectograms.shape[:-1].concatenate(mel_weight.shape[-1:]))\n log_mel_spectograms = tf.math.log(mel_spectrograms + 1e-6)\n\n results = {\n \"audio_spec\": tf.transpose(log_mel_spectograms) # of shape (num_mel_spec_bins, num_frames)\n }\n results.update(sample)\n return results\n pass","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:43:29.673907Z","iopub.execute_input":"2023-12-15T15:43:29.674257Z","iopub.status.idle":"2023-12-15T15:43:29.686106Z","shell.execute_reply.started":"2023-12-15T15:43:29.674223Z","shell.execute_reply":"2023-12-15T15:43:29.684553Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"@tf.function\ndef filter_tp(sample):\n \"\"\"\n\n :param sample: Processed dictionary from _parse_function\n :return: boolean, whether belongs to true positive or false positive\n \"\"\"\n return sample[\"is_tp\"] == 1\n pass","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:43:29.688289Z","iopub.execute_input":"2023-12-15T15:43:29.688789Z","iopub.status.idle":"2023-12-15T15:43:29.699977Z","shell.execute_reply.started":"2023-12-15T15:43:29.688740Z","shell.execute_reply":"2023-12-15T15:43:29.698662Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"@tf.function\ndef create_annot(sample):\n target = tf.one_hot(sample[\"species_id\"],\n 24,\n on_value=sample[\"is_tp\"],\n off_value=0)\n\n return {\n \"input\": sample[\"audio_spec\"], # obtained from creating spectograms from audio np arrays\n \"target\": tf.cast(target, tf.float32)\n }\n pass","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:43:29.701667Z","iopub.execute_input":"2023-12-15T15:43:29.702157Z","iopub.status.idle":"2023-12-15T15:43:29.711063Z","shell.execute_reply.started":"2023-12-15T15:43:29.702111Z","shell.execute_reply":"2023-12-15T15:43:29.709586Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"@tf.function\ndef toImage(logmelSpec):\n # expand one dimension axis to be treated as image\n image = tf.expand_dims(logmelSpec, axis=-1)\n image = tf.image.resize(image, (224, 512))\n image = tf.image.per_image_standardization(image)\n\n # no augmentation at this stage\n image = (image - tf.reduce_mean(image))/(tf.reduce_max(image) * tf.reduce_min(image)) * 255.0\n image = tf.image.grayscale_to_rgb(image)\n # image = cfg.model_params[\"preprocess\"](image)\n return image\n pass\n\n\n@tf.function\ndef preprocess_img(sample):\n image = toImage(sample[\"input\"])\n return image, sample[\"target\"]\n pass","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:43:29.712879Z","iopub.execute_input":"2023-12-15T15:43:29.713369Z","iopub.status.idle":"2023-12-15T15:43:29.726003Z","shell.execute_reply.started":"2023-12-15T15:43:29.713321Z","shell.execute_reply":"2023-12-15T15:43:29.724363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# let's extract the samples from the tfrecs and create our initial processed dataset\nspec_dataset = parsed_tfrecs_sample.filter(filter_tp).map(cut_audio).map(waveToSpec)\nimg_dataset = spec_dataset.map(create_annot).map(preprocess_img)","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:43:29.727801Z","iopub.execute_input":"2023-12-15T15:43:29.728349Z","iopub.status.idle":"2023-12-15T15:43:30.602093Z","shell.execute_reply.started":"2023-12-15T15:43:29.728301Z","shell.execute_reply":"2023-12-15T15:43:30.600921Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axs = plt.subplots(nrows=3, ncols=3, figsize=(20, 10))\nfor i, s in enumerate(spec_dataset.take(3)):\n axs[0, i].imshow(s['audio_spec'])\n axs[0, i].set_title(s['recording_id'].numpy().decode(\"UTF-8\") + \"->\" + \"s-\" +str(s[\"species_id\"].numpy()))\n LD.waveplot(s[\"audio_wav\"].numpy(), sr=SR, ax=axs[1, i])\n LD.specshow(s['audio_spec'].numpy(), x_axis=\"time\", y_axis=\"mel\", sr=SR, fmax=FMAX, fmin=FMIN, ax=axs[2, i], cmap=\"magma\")\n axs[2, i].add_patch(ptc.Rectangle(xy=(s[\"t_min\"], s[\"f_min\"]), height=s[\"f_max\"]-s[\"f_min\"], width=s[\"t_max\"]-s[\"t_min\"], fill=False))\n axs[2, i].text(s[\"t_min\"], s[\"f_min\"], \"s-\" +str(s[\"species_id\"].numpy()), horizontalalignment='left', verticalalignment='bottom', fontsize=16) \nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-12-15T15:43:30.603481Z","iopub.execute_input":"2023-12-15T15:43:30.603824Z","iopub.status.idle":"2023-12-15T15:43:34.449524Z","shell.execute_reply.started":"2023-12-15T15:43:30.603791Z","shell.execute_reply":"2023-12-15T15:43:34.448443Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Thank you for reading the notebook :)

","metadata":{}}]}