{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## BirdCLEF Call Identification Pre-Modeling\n---\n\n1. MetaData EDA\n2. Audio EDA\n3. Toy model with fastai","metadata":{"id":"xXfaQrny7ZAn"}},{"cell_type":"markdown","source":"### Acknowledgments\n\n---\n\n* [EDA kickstarter](https://www.kaggle.com/virajkadam/birdclef-exploratory-data-analysis)\n* [torch-librosa help](https://www.kaggle.com/whurobin/training-pipeline-in-pytorch-lightning/data)\n* [fastaudio help]( https://colab.research.google.com/drive/1hTRtTq3Tr9kgld0i78ao8gVrq_0yPTgW#scrollTo=UktmdDZ7wt8T)","metadata":{"id":"RqeZP79c7ZAo"}},{"cell_type":"code","source":"!pip uninstall fastai -y\n!pip install fastai==2.2.7\n!pip install fastaudio\n!pip install fastcore==1.3.19","metadata":{"id":"LIuaSqqX7ZAp","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Libraries\n\nimport os, random, math\nimport numpy as np\nimport pandas as pd \nimport geopandas as gpd \nimport matplotlib.pyplot as plt \nimport seaborn as sns \nexec(open('../input/plting/plt-apple-dark.py').read())\n\nfrom fastaudio.core.all import *\nfrom fastaudio.augment.all import *\nfrom fastai.torch_basics import *\nfrom fastai.vision.all import *\n\nimport torchaudio\nimport librosa.display\nimport librosa\nimport librosa.display\nfrom IPython.display import Audio\nfrom pathlib import Path\n","metadata":{"id":"dB6LQbW77ZAq","outputId":"215cea29-099f-48ab-a738-b094245d891c","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def seed_everything(seed=42):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.backends.cudnn.deterministic = True\n    \nseed_everything()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"torch.cuda.is_available()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1. MetaData EDA\n---\n\n* Basic EDA before modeling.\n* Where were the records taken?\n  * Are the records evenly distributed across space or unevenly?\n","metadata":{"id":"5LEIrFay7ZAr"}},{"cell_type":"code","source":"training_metadata_df=pd.read_csv('../input/birdclef-2021/train_metadata.csv')\nprint(f\"Len training data: {len(training_metadata_df.index)} \\n\\n\")\ntraining_metadata_df.sample(2)","metadata":{"id":"d5UCrHYd7ZAs","outputId":"7a7652b9-cfc8-4c39-eef7-3b4a9b791854","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f'Number of different species: {training_metadata_df.primary_label.nunique()}')\nprint(f\"Number of species > 100 audio records: {sum(training_metadata_df['common_name'].value_counts()<100)}\")\npercentage_low = 133/397*100\npercentage_low = \"{:.2f}\".format(percentage_low)\nprint(f\"At least {percentage_low}% of the records are infrequent\")","metadata":{"id":"wNlSP0kA7ZAs","outputId":"34fde19b-45b6-49ae-e32a-633be75f5dea","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f\"Minimum longitude: {training_metadata_df['longitude'].min()}\")\nprint(f\"Maximum longitude: {training_metadata_df['longitude'].max()}\")\nprint(f\"Minimum latitude: {training_metadata_df['latitude'].min()}\")\nprint(f\"Maximum latitude: {training_metadata_df['latitude'].max()}\")","metadata":{"id":"0WwsRIw37ZAt","outputId":"87193c93-17da-497b-f481-653f1638a13a","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = training_metadata_df.iloc[:, [0,3,4,8,9]]\ngdf = gpd.GeoDataFrame(\n    df, geometry=gpd.points_from_xy(df.longitude, df.latitude))\ngdf_1 = gdf[(gdf.latitude<35) & (gdf.longitude>-30)]\ngdf_2 = gdf[(gdf.latitude<10) & (gdf.longitude<-20)]\ngdf_3 = gdf[(gdf.latitude>=35) & (gdf.longitude>=-40)]\ngdf_4 = gdf[(gdf.latitude>=10) & (gdf.longitude<=-40)]\nworld = gpd.read_file(gpd.datasets.get_path('naturalearth_lowres'))\nax = world.plot(color='white', edgecolor='black')\n# We can now plot our ``GeoDataFrame``.\ngdf_1.plot(ax=ax, markersize=1)\ngdf_2.plot(ax=ax, markersize=1)\ngdf_3.plot(ax=ax, markersize=1)\ngdf_4.plot(ax=ax, markersize=1)\nplt.show()","metadata":{"id":"3iTpNwBL7ZAu","outputId":"8e9f3c58-fca9-4cbd-b8a8-0fbbb306165e","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"total = np.sum([len(gdf_1), len(gdf_2), len(gdf_3), len(gdf_4)])\nassert total == len(training_metadata_df.index)\nprint('The number of records for each quadrant of the earth: \\n' +\n       f'     -> {[len(gdf_1), len(gdf_3), len(gdf_2), len(gdf_4)]}\\n' +\n       f'     -> {[\"Af\", \"EU\", \"SA\", \"NA\"]}')","metadata":{"id":"VfrGi-Mh7ZAu","outputId":"76db976b-9d05-469d-9f65-bfb9f0e4c8f0","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### MetaData EDA observations\n---\n\nMost of the recordings are in North and South America, as well as West and North Europe. The rest of the map has varying sparcity.\n\nAs per the Kaggle competition intro:\n\n* \"Some bird species may have local call 'dialects,' so you may want to seek geographic diversity in your training data\". In addition, \"while some bird calls can be made year round, such as an alarm call, some are restricted to a specific season. You may want to seek temporal diversity in your training data.\" \n\nOur EDA confirms that this approach, seeking geospatial and temporal diversity, may be critical to preserve lower density types of audio records. For example, only 0.5% (0.005) of our data is from the Southeast quadrant of the globe.","metadata":{}},{"cell_type":"markdown","source":"## 2. Audio EDA\n\n---\n\n* Added subtle Pink noise and removed silence\n* Cropped to seven seconds\n* Converted to uint8 data to save memory\n* Loaded into kaggle: audio-flacs-birdclef21 dataset","metadata":{"id":"Q-hmIwy37ZAv"}},{"cell_type":"code","source":"audio_files = get_audio_files('../input/audio-flacs-birdclef21/audio_flac/')\naudio_files","metadata":{"id":"m9GDBNku7ZAv","outputId":"932f4132-95b4-4265-d451-755336510366","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y, sr = torchaudio.load(audio_files[0])\nprint(\"Sample rate:\", sr)\nprint(\"Signal Length:\", len(y))\nprint(\"Duration:\", len(y)/sr, \"seconds\")\ny = y.numpy()[0]\nprint(\"Signal: \", y)\nprint(\"Shape:\", y.shape)","metadata":{"id":"0uJDXjmt7ZAw","outputId":"9bb54d4c-1746-408c-db78-72022c219a62","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Anna's Hummingbird\n\nannhum, sr = torchaudio.load('../input/audio-flacs-birdclef21/audio_flac/annhum/XC57971.ogg.flac')\nplt.figure(figsize=(15, 5))\nlibrosa.display.waveplot(annhum.numpy()[0], sr=sr)\nannhum_audio = '../input/audio-flacs-birdclef21/audio_flac/annhum/XC57971.ogg.flac'\nAudio(annhum_audio)","metadata":{"id":"VZ5HtHcy7ZAx","outputId":"76f0b8bf-6d7e-4017-9e92-b0f75b0e0afe","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Bluejay\n\nblujay, sr = torchaudio.load('../input/audio-flacs-birdclef21/audio_flac/blujay/XC108404.ogg.flac')\nplt.figure(figsize=(15, 5))\nlibrosa.display.waveplot(blujay.numpy()[0], sr=sr)\nblujay_audio = '../input/audio-flacs-birdclef21/audio_flac/blujay/XC108404.ogg.flac'\nAudio(blujay_audio)","metadata":{"id":"7qj2Nadh7ZAy","outputId":"1978fff1-6a34-4386-9349-9527d15addd5","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Plot previous species\n\ngdf_blujay = gdf[gdf.primary_label == 'blujay']\ngdf_annhum = gdf[gdf.primary_label == 'annhum']\nworld = gpd.read_file(gpd.datasets.get_path('naturalearth_lowres'))\nax = world.plot(color='white', edgecolor='black')\ngdf_blujay.plot(ax=ax, markersize=0.3, label='bluejay')\ngdf_annhum.plot(ax=ax, markersize=0.3, label='annhum')\nplt.legend()\nplt.xlim([-150, -20])\nplt.ylim([10, 75])\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(nrows=2, ncols=1, sharex=True, figsize=(15, 10))\n\nsg0 = librosa.stft(annhum.numpy()[0])\nsg_mag, sg_phase = librosa.magphase(sg0)\nsg1 = librosa.feature.melspectrogram(S=sg_mag, sr=sr)\nsg2 = librosa.amplitude_to_db(sg1, ref=np.min)\nlibrosa.display.specshow(sg2, sr=sr, y_axis='mel', fmax=8000, x_axis='time', ax=ax[0])\nax[0].set(title='Annas hummingbird Mel spectrogram')\nax[0].label_outer()\n\nsg0 = librosa.stft(blujay.numpy()[0])\nsg_mag, sg_phase = librosa.magphase(sg0)\nsg1 = librosa.feature.melspectrogram(S=sg_mag, sr=sr)\nsg2 = librosa.amplitude_to_db(sg1, ref=np.min)\nlibrosa.display.specshow(sg2, sr=sr, y_axis='mel', fmax=8000, x_axis='time', ax=ax[1])\nax[1].set(title='Bluejay Mel spectrogram')\nax[1].label_outer()","metadata":{"id":"BSW7DqpQ7ZAy","outputId":"c3847b82-564d-45d2-f8ff-f4fe5d14e0f7","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 3. Build a toy model with fastai to experiment\n---\n\n* Audio to melspec batch transform\n* densenet121, as per [last comp's winning solution](https://www.kaggle.com/c/birdsong-recognition/discussion/183208)","metadata":{}},{"cell_type":"code","source":"audio_files","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"training_metadata_df.sample(2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class AudioConfig:\n    \"\"\"\n    Custom `AudioToSpec` transform for birdclef \n    \"\"\"\n    birds = config_from_func(\n        transforms.MelSpectrogram,\n        \"Voice\",\n        mel=\"True\",\n        to_db=\"True\",\n        f_min=50.0,\n        f_max=8000.0,\n        n_fft=2048,\n        n_mels=64,\n        hop_length=int(2048 // 4)\n    )","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cfg = AudioConfig.birds()\n\nbatch_tfms = [AudioToSpec.from_cfg(cfg)]\n\nget_y = lambda x: re.split('[/]', x.as_posix())[-2]\n\naudio_db = DataBlock(blocks = (AudioBlock, CategoryBlock),\n                     get_items = get_audio_files,\n                     splitter = RandomSplitter(),\n                     batch_tfms = batch_tfms, #augments wouldn't work on batch\n                     get_y=get_y)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dbunch = audio_db.dataloaders('../input/audio-flacs-birdclef21/audio_flac', bs=128)\ndbunch.show_batch(figsize=(10, 5))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"audio_db.summary('../input/audio-flacs-birdclef21/audio_flac')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from fastai.callback.data import CudaCallback\n\nlearn = cnn_learner(dbunch, \n            densenet121,\n            n_in=1,\n            loss_func=CrossEntropyLossFlat(),\n            metrics=[error_rate],\n            cbs=[CudaCallback]).to_fp16()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.lr_find() ","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.fit_one_cycle(3, 6e-3)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.unfreeze()\nlearn.lr_find() ","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.fit_one_cycle(3, lr_max=slice(1e-6,1e-4))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Looks like training after unfreezing doesn't move the needle much","metadata":{}},{"cell_type":"code","source":"preds_list = list()\nfor i in range(0, len(targs)):\n    preds_list.append(preds[i].argmax().item())\n\npreds_tensor = TensorCategory(preds_list)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"f1score = F1Score(average='micro')\npreds, targs = learn.tta()\nprint(f'Micro average F1 validation: {f1score(preds_tensor, targs).item()}')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"np.savetxt(\"raw_pred_probabilities.csv\", np.array(preds), delimiter=\",\")\nnp.savetxt(\"final_preds.csv\", np.array(preds_list), delimiter=\",\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}