{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## 📢 **Updates**\n- **05 Jun 2021**: Table are logged into a run with bigger subset of datasets\n- **31 May 2021**: Table are logged into an artifact with small subset of datasets","metadata":{}},{"cell_type":"markdown","source":"## 🎯 **Goal**\n[Rainforest Connection Species Audio Detection competition](https://www.kaggle.com/c/rfcx-species-audio-detection/overview) contains thousands of audio clips with both species labels and their localized regions.  \nUtilising the recently released feature (`wandb.Table`) by [Weight and Bias](https://wandb.ai/site), This notebook serves the followin purposes:\n- Offers an off-the-shelf tabular tool to explore the datasets\n- Walk you through how to create an awesome tool like this using W&B","metadata":{}},{"cell_type":"markdown","source":"## 📚 **References**\n- [W&B Dataset and Predictions Viz Demo](https://colab.research.google.com/github/wandb/examples/blob/master/colabs/dsviz/W%26B_Dataset_and_Predictions_Viz_Demo.ipynb)\n- [Visualize Audio Data in W&B](https://wandb.ai/stacey/cshanty/reports/Visualize-Audio-Data-in-W-B--Vmlldzo1NDMxMDk)\n\nAdditionally, I would like to take this chance to thank [@ayuraj](https://www.kaggle.com/ayuraj) for introducing W&B to me and for all the amazing notebooks you have created!","metadata":{}},{"cell_type":"markdown","source":"## 🔍 **Try It Out Now!**\nThe tool is freely hosted in W&B and you can check it out here:  \n[https://wandb.ai/alexlauwh/RFCX-EDA-v2?workspace=user-alexlauwh](https://wandb.ai/alexlauwh/RFCX-EDA-v2?workspace=user-alexlauwh)\n\nThe interface offers the following basic functions:\n- Hear the localized audio for each species (include true positive, false positive and each songtype)  \n- Inspect its associated melspectrogram\n- Identify the time & frequency range from the melspectrogram associated to the species\n\n\nBelow is a snapshot of the interface:  \n![](https://imgur.com/HOcgj0V.png)\n\n\nIn addition to the above functions, you can also:  \n- filter sample by simple criteria (e.g. filter by `species_id = 12` as shown)  \n![](https://imgur.com/f8D9cMP.png)\n\n- aggregate samples for high-level analysis (e.g. group by `species_id` as shown)  \n![](https://imgur.com/LngQqlz.png)\n\nI Here I just highlighted key features that I think are useful. You could refer to [the documentation](https://docs.wandb.ai/guides/data-vis) for its full functionalities.","metadata":{}},{"cell_type":"markdown","source":"## ❓ **How To Create a Tabular Tool Like This**\nRun the code below to reproduce the tabular tool using W&B, but before that there are a few steps you need to complete:\n1. Apply a W&B account and get your W&B API key from **\"User Settings\"**  \n![](https://i.imgur.com/PY0Ywuh.png)\n2. Create an environment variable `wandb_api` in this kernel for the API key. To do this navigate to the panel below by **\"Add-ons\" >> \"Secrets\"**  \n![](https://imgur.com/633GGXU.png)\n\nThe code will do the following:\n1. Create a project and a run in your account\n2. Create a `wandb.Table` object\n3. Log the `wandb.Table` object into your run","metadata":{}},{"cell_type":"code","source":"!pip install wandb --upgrade","metadata":{"execution":{"iopub.status.busy":"2021-05-31T15:44:28.266411Z","iopub.execute_input":"2021-05-31T15:44:28.266875Z","iopub.status.idle":"2021-05-31T15:44:41.864556Z","shell.execute_reply.started":"2021-05-31T15:44:28.266743Z","shell.execute_reply":"2021-05-31T15:44:41.863135Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import tempfile\nfrom pathlib import Path\nfrom typing import Tuple\n\nimport wandb\nfrom kaggle_secrets import UserSecretsClient\n\nimport librosa\nimport librosa.display\n\nfrom tqdm import tqdm\nfrom PIL import Image\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as patches\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-05-31T15:44:41.866457Z","iopub.execute_input":"2021-05-31T15:44:41.866854Z","iopub.status.idle":"2021-05-31T15:44:45.035299Z","shell.execute_reply.started":"2021-05-31T15:44:41.866802Z","shell.execute_reply":"2021-05-31T15:44:45.034345Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# WANDB CONFIG\n# define your prefered project and run name\nWANDB_PROJECT = 'RFCX-EDA-v2'\nWANDB_RUN_NAME = 'RFCX-EDA-v2'\nWANDB_JOB_TYPE = 'EDA'\n# upload subset of data to artifact, instead of all data (avoid memory crash)\nSAMPLE_N_PER_GROUP = 20\n\n# DATA CONFIG\nDATA_DIR = Path('/kaggle/input/rfcx-species-audio-detection')\nTP_PATH = DATA_DIR/ 'train_tp.csv'\nFP_PATH = DATA_DIR/ 'train_fp.csv'\n\n# AUDIO/ PLOTTING CONFIG\nSR = 48000\nFMAX = int(SR/2)\nFMIN = 0\nLIBROSA_CONFIG = {\n    'sr': SR,\n    'n_fft': 2048,\n    'hop_length': 512,\n    'fmin': FMIN,\n    'fmax': FMAX\n}\nDISPLAY_CONFIG = {\n    'sr': SR, \n    'fmin': FMIN,\n    'fmax': FMAX, \n    'cmap': 'viridis'\n}\nRECTANGLE_CONFIG = {\n    'linewidth': 1., \n    'edgecolor': 'yellow', \n    'facecolor': 'yellow', \n    'alpha': 0.2\n}","metadata":{"execution":{"iopub.status.busy":"2021-05-31T15:44:45.037164Z","iopub.execute_input":"2021-05-31T15:44:45.037628Z","iopub.status.idle":"2021-05-31T15:44:45.044429Z","shell.execute_reply.started":"2021-05-31T15:44:45.037578Z","shell.execute_reply":"2021-05-31T15:44:45.043384Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class RowGenerator:\n    def __init__(self, df: pd.DataFrame, data_dir: Path=DATA_DIR, sr: str=SR):\n        self.df = df\n        self.data_dir = data_dir\n        self.sr = sr\n        \n    def __getitem__(self, idx) -> Tuple:\n        row = self.df.loc[idx]\n        cut_audio, start_t = self.read_cut_audio(row)\n        \n        # get audio waveplot\n        wandb_audio = wandb.Audio(cut_audio, sample_rate=self.sr)\n        # get melspectrogram with localized box\n        offset_t = row.t_min - (start_t/self.sr)\n        pt = (offset_t, row.f_min)\n        width = row.t_max-row.t_min\n        height = row.f_max-row.f_min\n        melspec_fig = self.render_melspec_with_box_plot(cut_audio, pt,\n                                                        width, height)\n        melspec_wandb_image = self.fig_to_wandb_image(melspec_fig)\n        # other metadata\n        recording_id = row.recording_id\n        species_id = row.species_id\n        songtype_id = row.songtype_id\n        label_type = row.label_type\n        \n        out = (recording_id, species_id, songtype_id,\n               label_type, wandb_audio, melspec_wandb_image)\n        return out\n            \n    def __len__(self):\n        return self.df.shape[0]\n    \n    def read_cut_audio(self, row: pd.Series) -> Tuple[np.ndarray, int]:        \n        fn = self.data_dir/ f'train/{row.recording_id}.flac'\n        assert fn.is_file()\n        audio, sr = librosa.load(fn, sr=self.sr)\n        t_min, t_max = int(row.t_min*self.sr), int(row.t_max*self.sr)\n        t_min = int(max(t_min-self.sr, 0))\n        t_max = int(min(t_max+self.sr, 60*self.sr))\n        return audio[t_min:t_max], t_min\n    \n    @staticmethod\n    def fig_to_wandb_image(fig: plt.Figure) -> wandb.Image:\n        with tempfile.NamedTemporaryFile(suffix='.png') as tmpfile:\n            fig.savefig(tmpfile.name)\n            img = Image.open(tmpfile.name)\n            wandb_img = wandb.Image(img)\n            # prevent figure being displayed in notebook\n            plt.close()\n        return wandb_img\n    \n    @staticmethod\n    def render_melspec_with_box_plot(audio: np.ndarray, pt: Tuple[float, float], width: float, height: float) -> plt.Figure:\n        \"\"\" \n        render melspectrogram to visible image \n        ref: https://www.kaggle.com/gpreda/explore-the-rainforest-soundscape\n        \"\"\"\n        fig, ax = plt.subplots(figsize=(16, 9))\n        spec = librosa.feature.melspectrogram(audio, **LIBROSA_CONFIG)\n        dbs = librosa.amplitude_to_db(abs(spec))\n        librosa.display.specshow(dbs, x_axis='time', y_axis='mel',\n                                 **DISPLAY_CONFIG)\n        rec = patches.Rectangle(pt, width=width, height=height,\n                                **RECTANGLE_CONFIG)\n        ax.add_patch(rec)\n        plt.tight_layout()\n        plt.colorbar()\n        return fig\n    \n    @staticmethod\n    def render_waveplot(audio: np.ndarray) -> plt.Figure:\n        fig, ax = subplots(figsize=(16, 9))\n        ax = librosa.display.waveplot(audio, sr=self.sr)\n        return fig","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2021-05-31T15:44:45.046893Z","iopub.execute_input":"2021-05-31T15:44:45.04719Z","iopub.status.idle":"2021-05-31T15:44:45.069425Z","shell.execute_reply.started":"2021-05-31T15:44:45.047163Z","shell.execute_reply":"2021-05-31T15:44:45.068463Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tp_df = pd.read_csv(TP_PATH)\ntp_df['label_type'] = 'TP'\nfp_df = pd.read_csv(FP_PATH)\nfp_df['label_type'] = 'FP'\ndf = pd.concat([tp_df, fp_df]).reset_index(drop=True)\ndf = df.groupby(by=['species_id', 'songtype_id', 'label_type']).sample(SAMPLE_N_PER_GROUP).reset_index(drop=True)","metadata":{"execution":{"iopub.status.busy":"2021-05-31T15:45:51.779111Z","iopub.execute_input":"2021-05-31T15:45:51.779638Z","iopub.status.idle":"2021-05-31T15:45:51.814448Z","shell.execute_reply.started":"2021-05-31T15:45:51.779591Z","shell.execute_reply":"2021-05-31T15:45:51.813137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# read env var set in kernel\nuser_secrets = UserSecretsClient()\nwandb_api = user_secrets.get_secret(\"wandb_key\")\nwandb.login(key=wandb_api)\nrun = wandb.init(project=WANDB_PROJECT, name=WANDB_RUN_NAME, job_type=WANDB_JOB_TYPE)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#filter_df = df[df.label_type == 'TP'].reset_index(drop=True)\nprocessor = RowGenerator(df)\ncolumns = ['recording_id', 'species_id', 'songtype_id', 'label_type', 'audio', 'melspectrogram']\ntable = wandb.Table(columns)\n\nsample_n = df.shape[0]\nprint(f'Creating wandb.Table of {sample_n} entries')\nfor idx in tqdm(range(sample_n)):\n    row_tuple = processor[idx];\n    table.add_data(*row_tuple)\n    if idx % 500 == 0:\n        # this line is so noisy\n        plt.clf()\n        print(f'Completed {idx+1} rows')\n        \nrun.log({'EDA_Table': table})        \nprint('Run completed')","metadata":{},"execution_count":null,"outputs":[]}]}