{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Possibly everything you need to know about the contest\n\n![Header](https://storage.googleapis.com/kaggle-competitions/kaggle/21669/logos/header.png)\n\nThis notebook is created to understand what needs to be done with the dataset obtained in the contest. I have tried to cover a few possible aspects and explained them in simple language. What I have tried to cover is mentioned in the index below:\n\n## Content\n\n1. [Evaluation metrics](https://www.kaggle.com/thanatoz/everything-you-need-to-know-about-the-contest?scriptVersionId=47211475#Evaluation-metrics)\n    * [Explaination](https://www.kaggle.com/thanatoz/everything-you-need-to-know-about-the-contest?scriptVersionId=47211475#Explaination)\n2. [Looking into the dataset](https://www.kaggle.com/thanatoz/everything-you-need-to-know-about-the-contest?scriptVersionId=47211475#Looking-into-the-dataset)\n    * [The .csv files](https://www.kaggle.com/thanatoz/everything-you-need-to-know-about-the-contest?scriptVersionId=47211475#The-.csv-files)\n    * [Train test folders](https://www.kaggle.com/thanatoz/everything-you-need-to-know-about-the-contest?scriptVersionId=47211475#Train-test-folders)\n    * [What is FLAC?](https://www.kaggle.com/thanatoz/everything-you-need-to-know-about-the-contest?scriptVersionId=47211475#What-is-FLAC?)\n    * [Visualizing the Audio file](https://www.kaggle.com/thanatoz/everything-you-need-to-know-about-the-contest?scriptVersionId=47211475#Visualizing-the-Audio-file)\n    * [Loading the FLAC files](https://www.kaggle.com/thanatoz/everything-you-need-to-know-about-the-contest?scriptVersionId=47211475#Loading-the-flac-files)\n    * [Plotting the Audio files](https://www.kaggle.com/thanatoz/everything-you-need-to-know-about-the-contest?scriptVersionId=47211475#Plotting-the-Audio-files)\n3. [References](https://www.kaggle.com/thanatoz/everything-you-need-to-know-about-the-contest?scriptVersionId=47211475#References)\n    \nI'll keep adding more and more data into this file as it clicks me, but till then you can share what else could be added into the file in the comment section."},{"metadata":{},"cell_type":"markdown","source":"## Evaluation metrics\n\n[Label ranking average precision (LRAP)](https://scikit-learn.org/stable/modules/model_evaluation.html#label-ranking-average-precision) measures the average precision of predictive model instead of using precision and recall whose values ranges from $0\\lt x\\le1$. To simplify, it answers the question that for each of the given samples what percents of the higher-ranked labels were true labels.\n\nFormally, given a binary indicator matrix of the ground truth labels $ y \\in \\left\\{0, 1\\right\\}^{n_\\text{samples} \\times n_\\text{labels}} $ and the score associated with each label $\\hat{f} \\in \\mathbb{R}^{n_\\text{samples} \\times n_\\text{labels}}$ the average precision is defined as\n$$ LRAP(y, \\hat{f}) = \\frac{1}{n_{\\text{samples}}}\n  \\sum_{i=0}^{n_{\\text{samples}} - 1} \\frac{1}{||y_i||_0}\n  \\sum_{j:y_{ij} = 1} \\frac{|{L}_{ij}|}{\\text{rank}_{ij}} $$\n  \nwhere ${L}_{ij} = \\left\\{k: y_{ik} = 1, \\hat{f}_{ik} \\geq \\hat{f}_{ij} \\right\\}$, $\\text{rank}_{ij} = \\left|\\left\\{k: \\hat{f}_{ik} \\geq \\hat{f}_{ij} \\right\\}\\right|$, $|\\cdot|$ computes the cardinality of the set (i.e., the number of elements in the set), and $||\\cdot||_0$ is the $\\ell_0$ “norm” (which computes the number of nonzero elements in a vector).\n\n  "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nfrom sklearn.metrics import label_ranking_average_precision_score\n\ny_true = np.array([[1, 0, 0], [0, 0, 1]])\ny_score = np.array([[0.75, 0.5, 1], [1, 0.2, 0.1]])\nlabel_ranking_average_precision_score(y_true, y_score)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Explaination\n\nIn order to simplify the above calculation, we can break it down as follows:\n1. In the first sample, the y_true is `[1, 0, 0]` and the y_score is `[0.75, 0.5, 1]` which means that the class A is given $2^{nd}$ rank. This makes the value of $ {\\sum_{j:y_{ij} = 1}} \\frac{|{L}_{ij}|}{\\text{rank}_{ij}}$ to be 1/2 = 0.5.\n\n2. In the second sample, the y_true is `[0, 0, 1]` and the y_score is `[1, 0.2, 0.1]` which means that the class C is given $3^{rd}$ rank. This gives the value of $ {\\sum_{j:y_{ij} = 1}} \\frac{|{L}_{ij}|}{\\text{rank}_{ij}}$ to be 1/3 = 0.333333333\n\nNow we calculate the average of all the samples (2 in our case) and thus our final result will be $\\frac{1}{2}*(0.5+0.33333333) = 0.416666666$"},{"metadata":{},"cell_type":"markdown","source":"## Looking into the dataset"},{"metadata":{"trusted":true},"cell_type":"code","source":"import pandas as pd\nimport matplotlib\nfrom matplotlib import pyplot as plt\nimport sklearn\nimport seaborn as sns\nimport plotly.figure_factory as ff\n\nmatplotlib.rcParams['figure.figsize'] = (12.0, 6.0)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### The .csv files\n\nThe dataset contains 2 csv files:\n> 1. **train_tp.csv** - training data of true positive species labels, with corresponding time localization\n> 2. **train_fp.csv** - training data of false positives species labels, with corresponding time localization\n\nWhere the information of every column is as follows:\n> 1. **recording_id** - The id of file containing the recording\n> 2. **species_id** - The id of the specie\n> 3. **songtype_id** - The id for songtype\n> 4. **t_min, t_max** - Contains the start and end timing of the annotated signal\n> 5. **f_min, f_max** - Contains the lower frequency and upper of annotated signal\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"tp_data=pd.read_csv('/kaggle/input/rfcx-species-audio-detection/train_tp.csv')\nfp_data=pd.read_csv('/kaggle/input/rfcx-species-audio-detection/train_fp.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tp_data.tail()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fp_data.tail()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_counts(feature:str, dataframe:pd.DataFrame, kind:str=\"bar\"):\n    dataframe[feature].value_counts().plot(kind=kind)\n    \nplot_counts('species_id', tp_data)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"plot_counts(\"songtype_id\", tp_data)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Train test folders\n\nIn the train test folders of the dataset, we are presented with flac files with labels in corresponding csv files. The number of files that we have are:"},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\n\nprint(f'Number of files in Train folder are: {len(os.listdir(\"../input/rfcx-species-audio-detection/train/\"))}, \\\nand test folder are {len(os.listdir(\"../input/rfcx-species-audio-detection/test/\"))}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## What is FLAC?\n\nTo simply state, FLAC files are a lossless encoding format for storing audio. FLAC ‘Quality’ is determined by the ‘sample rate’, and the ‘bit depth’.  Sample rates start at 22,050hz, 32,000hz, 44,1000hz, 48,000hz, 88,200, 96,000hz and go up to 192,000hz, and bit size can generally range from 8 to 32 bits. \n![flac vs mp3](https://www.off-the-beat.com/wp-content/uploads/2020/02/FLAC-vs-MP3-.jpg)\nIf you want to read more about FLAC vs MP3 encoding, here is a wonderful blog that you can refer: https://www.off-the-beat.com/flac-vs-mp3/"},{"metadata":{},"cell_type":"markdown","source":"## Visualizing the Audio file\n\nRefering to Allen Downey's talk on [Basics of sound processing](https://www.youtube.com/watch?v=0ALKGR0I5MA) at SciPy 2015 and [his presentation](https://docs.google.com/presentation/d/1zzgNu_HbKL2iPkHS8-qhtDV20QfWt9lC3ZwPVZo8Rw0/pub?start=false&loop=false&delayms=3000&slide=id.g5a7a9806e_0_14), we can understand audio files as wave files as follows:\n\n![Amplitudes](https://miro.medium.com/max/1400/1*akRbhl8739UEDuKHkOUR1Q.png)"},{"metadata":{},"cell_type":"markdown","source":"### Loading the flac files\n\nFlac files, just like mp3 files could be easily loaded using the _librosa_ library. _Librosa_ is the ultimate library for most of the audio processing requirements that you want to carry out. Lets start by calling the library and loading a file."},{"metadata":{"trusted":true},"cell_type":"code","source":"import librosa\nfrom librosa import display","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"audio_path = f'../input/rfcx-species-audio-detection/train/{tp_data[\"recording_id\"].values[0]}.flac'\nx, sr = librosa.load(audio_path)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"where _x_ is the data from the file and _sr_ is the sampling rate of the file. The function returns the numpy array with timestamps and the default sampling rate of 22KHz (Sampling rate: number of samples per second measured in Hz or KHz)."},{"metadata":{"trusted":true},"cell_type":"code","source":"print(x.shape, sr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can also change the sampling rate of the file by some arbitary value as follows"},{"metadata":{"trusted":true},"cell_type":"code","source":"librosa.load(audio_path, sr=44100)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import IPython.display as ipyd","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ipyd.Audio(audio_path)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Plotting the Audio files\n\nLibrosa gives us a huge options to work with. So Let's start by plotting the amplitude envelope of a waveform."},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(14, 5))\ndisplay.waveplot(x, sr=sr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Fourier transform**\n\nThe \"Fast Fourier Transform\" (FFT) is an important measurement method in the science of audio and acoustics measurement. It converts a signal into individual spectral components and thereby provides frequency information about the signal. FFTs are used for fault analysis, quality control, and condition monitoring of machines or systems."},{"metadata":{"trusted":true},"cell_type":"code","source":"n_fft = 2048\nD = np.abs(librosa.stft(x[:n_fft], n_fft=n_fft, hop_length=n_fft+1))\nplt.plot(D);","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"hop_length = 512\nD = np.abs(librosa.stft(x, n_fft=n_fft,  hop_length=hop_length))\ndisplay.specshow(D, sr=sr, x_axis='time', y_axis='linear')\nplt.colorbar()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"1. **Spectrogram**\n\nA spectrogram is a visual representation of the spectrum of frequencies of sound or other signals as they vary with time. Spectrograms are sometimes called sonographs, voiceprints, or voicegrams. When the data is represented in a 3D plot, they may be called waterfalls. In 2-dimensional arrays, the first axis is frequency while the second axis is time. We can display a spectrogram using. librosa.display.specshow."},{"metadata":{"trusted":true},"cell_type":"code","source":"X = librosa.stft(x)\nXdb = librosa.amplitude_to_db(abs(X))\nplt.figure(figsize=(14, 5))\ndisplay.specshow(Xdb, sr=sr, x_axis='time', y_axis='hz')\nplt.colorbar()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The vertical axis shows frequencies (from 0 to 10kHz), and the horizontal axis shows the time of the clip. Since we see that all action is taking place at the bottom of the spectrum, we can convert the frequency axis to a logarithmic one."},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(14, 5))\ndisplay.specshow(Xdb, sr=sr, x_axis='time', y_axis='log')\nplt.colorbar()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"2. **Feature extraction**\n\nEvery audio signal consists of many features. However, we must extract the characteristics that are relevant to the problem we are trying to solve. The process of extracting features to use them for analysis is called feature extraction. Let us study about few of the features in detail."},{"metadata":{},"cell_type":"markdown","source":"**Zero Crossing Rate**\n\nThe zero crossing rate is the rate of sign-changes along a signal, i.e., the rate at which the signal changes from positive to negative or back. Let us calculate the zero crossing rate for our example audio clip."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Zooming in the amplitute envelope\nn0 = 5000\nn1 = 5100\nplt.figure(figsize=(14, 5))\nplt.plot(x[n0:n1])\nplt.grid()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"zero_crossings = librosa.zero_crossings(x[n0:n1], pad=False)\nprint(sum(zero_crossings))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Spectral Centroid**\n\nIt indicates where the ”centre of mass” for a sound is located and is calculated as the weighted mean of the frequencies present in the sound. Consider two songs, one from a blues genre and the other belonging to metal."},{"metadata":{"trusted":true},"cell_type":"code","source":"spectral_centroids = librosa.feature.spectral_centroid(x, sr=sr)[0]\nspectral_centroids.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Computing the time variable for visualization\nframes = range(len(spectral_centroids))\nt = librosa.frames_to_time(frames)\n# Normalising the spectral centroid for visualisation\ndef normalize(x, axis=0):\n    return sklearn.preprocessing.minmax_scale(x, axis=axis)\n#Plotting the Spectral Centroid along the waveform\nlibrosa.display.waveplot(x, sr=sr, alpha=0.4)\nplt.plot(t, normalize(spectral_centroids), color='r')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Mel-Frequency Cepstral Coefficients**\n\nThe Mel frequency cepstral coefficients (MFCCs) of a signal are a small set of features (usually about 10–20) which concisely describe the overall shape of a spectral envelope. It models the characteristics of the human voice."},{"metadata":{"trusted":true},"cell_type":"code","source":"mfccs = librosa.feature.mfcc(x, sr=sr)\nprint(mfccs.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Displaying  the MFCCs:\ndisplay.specshow(mfccs, sr=sr, x_axis='time')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## References\n\n1. [Explore the rainforest soundscape](https://www.kaggle.com/gpreda/explore-the-rainforest-soundscape)\n2. [EDA and Audio Processing with Python](https://www.kaggle.com/parulpandey/eda-and-audio-processing-with-python/?scriptVersionId=37933197#data)\n3. [Allen Downey SciPy2015 presentation](https://docs.google.com/presentation/d/1zzgNu_HbKL2iPkHS8-qhtDV20QfWt9lC3ZwPVZo8Rw0/pub?start=false&loop=false&delayms=3000&slide=id.p)"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}