{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# FORK of [ https://www.kaggle.com/stefankahl/birdclef2021-model-training ]. ","metadata":{}},{"cell_type":"markdown","source":"**Inference notebook** [ https://www.kaggle.com/virajkadam/birdclef-inference ]\n\n**Reference notebook** [ https://www.kaggle.com/code/faizyrana/birdclef2021-bird-species-recognition-using-cnn ]","metadata":{}},{"cell_type":"markdown","source":"# Treinando modelo\n\nIn this notebook, we will train our first model and apply this model to a soundscape. We will keep the amount of training samples, species and soundscapes to a minimum to keep the execution time as short as possible. Remember, this is only a sample implementation, feel free to explore your own workflow.\n\nThese are the steps that we will cover:\n\n\n* selecionar arquivos de audio que queremos usar para treinamento\n* extrair espectrogramas dos arquivos e salvá-los no diretório de trabalho\n* carregar amostras selecionadas em um largo banco de dados in-memory  \n* construir uma CNN simples para iniciantes  \n* treinar o modelo  \n* aplicar o modelo para selecionar o soundscape e verificar os resultados\n\n\n# 1. Configurações e importações\n\nVamos começar importando bibliotecas e setando algumas configurações.","metadata":{}},{"cell_type":"code","source":"import os\n\nimport warnings\nwarnings.filterwarnings(action='ignore')\n\nimport pandas as pd\nimport librosa\nimport numpy as np\nimport pickle\n\nfrom sklearn.utils import shuffle\nfrom PIL import Image\nfrom tqdm import tqdm\nimport matplotlib.pyplot as plt\n\nimport tensorflow as tf\nfrom tensorflow_addons.metrics import F1Score","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-03-14T05:01:33.723993Z","iopub.execute_input":"2023-03-14T05:01:33.724526Z","iopub.status.idle":"2023-03-14T05:01:43.327836Z","shell.execute_reply.started":"2023-03-14T05:01:33.724477Z","shell.execute_reply":"2023-03-14T05:01:43.326651Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Global vars\nRANDOM_SEED = 1337\nSAMPLE_RATE = 32000\nSIGNAL_LENGTH = 5 # seconds\nSPEC_SHAPE = (48, 128) # height x width\nFMIN = 500\nFMAX = 12500\n# MAX_AUDIO_FILES = 10000\nEPOCHS=50","metadata":{"execution":{"iopub.status.busy":"2023-03-14T05:01:46.626026Z","iopub.execute_input":"2023-03-14T05:01:46.626480Z","iopub.status.idle":"2023-03-14T05:01:46.632316Z","shell.execute_reply.started":"2023-03-14T05:01:46.626441Z","shell.execute_reply":"2023-03-14T05:01:46.631098Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2. Preparação dos dados\n\nPegando arquivos de audio com nota >= 4 e tendo mais que 175 amostras.","metadata":{}},{"cell_type":"code","source":"# Code adapted from: \n# https://www.kaggle.com/frlemarchand/bird-song-classification-using-an-efficientnet\n# Make sure to check out the entire notebook.\n\n# Carregando arquivo de metadata\ntrain = pd.read_csv('../input/birdclef-2021/train_metadata.csv',)\n\n# Limite de número de amostras e classes a ser tratado\n# Primeiramente, usar somente amostras de alta qualidade\ntrain = train.query('rating>=4')\n\n# Segundo, assume-se que pássaros com as amostras mais treinadas são também as mais comuns\n# Vamos colocar que a espécie precisa de pelo menos 200 áudios com avaliação acima de 4 para ser considerada comum\nbirds_count = {}\nfor bird_species, count in zip(train.primary_label.unique(), \n                               train.groupby('primary_label')['primary_label'].count().values):\n    birds_count[bird_species] = count\nmost_represented_birds = [key for key,value in birds_count.items() if value >= 175] \n\nTRAIN = train.query('primary_label in @most_represented_birds')\nLABELS = sorted(TRAIN.primary_label.unique())\n\n# Let's see how many species and samples we have left\nprint('NUMBER OF SPECIES IN TRAIN DATA:', len(LABELS))\nprint('NUMBER OF SAMPLES IN TRAIN DATA:', len(TRAIN))\nprint('LABELS:', most_represented_birds)","metadata":{"execution":{"iopub.status.busy":"2023-03-14T05:01:54.939891Z","iopub.execute_input":"2023-03-14T05:01:54.940337Z","iopub.status.idle":"2023-03-14T05:01:55.545636Z","shell.execute_reply.started":"2023-03-14T05:01:54.940301Z","shell.execute_reply":"2023-03-14T05:01:55.544294Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# salvando os nomes \nwith open('LABELS.pkl','wb') as f:\n    pickle.dump(LABELS,f)","metadata":{"execution":{"iopub.status.busy":"2023-03-14T05:02:00.569453Z","iopub.execute_input":"2023-03-14T05:02:00.569913Z","iopub.status.idle":"2023-03-14T05:02:00.576292Z","shell.execute_reply.started":"2023-03-14T05:02:00.569876Z","shell.execute_reply":"2023-03-14T05:02:00.574996Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Extraindo espectrogramas","metadata":{}},{"cell_type":"code","source":"# mistura os dados de treinamento e limita o numero de arquivos de audio para MAX_AUDIO_FILES\nTRAIN = shuffle(TRAIN, random_state=RANDOM_SEED)\n\n# definição da função que corta um arquivo de audio\n# extração de espectrogramas e salva eles\ndef get_spectrograms(filepath, primary_label, output_dir):\n    \n    # abre o arquivo com librosa, limitando aos primeiros 15 s\n    sig, rate = librosa.load(filepath, sr=SAMPLE_RATE, offset=None, duration=15)\n    \n    # divide o áudio em pedaços de 5 s\n    sig_splits = []\n    for i in range(0, len(sig), int(SIGNAL_LENGTH * SAMPLE_RATE)):\n        split = sig[i:i + int(SIGNAL_LENGTH * SAMPLE_RATE)]\n\n        # final do arquivo de sinal\n        if len(split) < int(SIGNAL_LENGTH * SAMPLE_RATE):\n            break\n        \n        sig_splits.append(split)\n        \n    # extração de spectrogramas mel para cada pedaço de áudio\n    s_cnt = 0\n    saved_samples = []\n    for chunk in sig_splits:\n        \n        hop_length = int(SIGNAL_LENGTH * SAMPLE_RATE / (SPEC_SHAPE[1] - 1))\n        mel_spec = librosa.feature.melspectrogram(y=chunk, \n                                                  sr=SAMPLE_RATE, \n                                                  n_fft=1024, \n                                                  hop_length=hop_length, \n                                                  n_mels=SPEC_SHAPE[0], \n                                                  fmin=FMIN, \n                                                  fmax=FMAX)\n    \n        mel_spec = librosa.power_to_db(mel_spec, ref=np.max) \n        \n        # normalização\n        mel_spec -= mel_spec.min()\n        mel_spec /= mel_spec.max()\n        \n        # salvar como arquivo de imagem\n        save_dir = os.path.join(output_dir, primary_label)\n        if not os.path.exists(save_dir):\n            os.makedirs(save_dir)\n        save_path = os.path.join(save_dir, filepath.rsplit(os.sep, 1)[-1].rsplit('.', 1)[0] + \n                                 '_' + str(s_cnt) + '.png')\n        im = Image.fromarray(mel_spec * 255.0).convert(\"L\")\n        im.save(save_path)\n        \n        saved_samples.append(save_path)\n        s_cnt += 1\n        \n        \n    return saved_samples\n\nprint('FINAL NUMBER OF AUDIO FILES IN TRAINING DATA:', len(TRAIN))","metadata":{"execution":{"iopub.status.busy":"2023-03-14T05:02:36.659839Z","iopub.execute_input":"2023-03-14T05:02:36.660317Z","iopub.status.idle":"2023-03-14T05:02:36.680241Z","shell.execute_reply.started":"2023-03-14T05:02:36.660271Z","shell.execute_reply":"2023-03-14T05:02:36.679042Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# processa arquivos de audio e extrai amostras de treino\ninput_dir = '../input/birdclef-2021/train_short_audio/'\noutput_dir = '../working/melspectrogram_dataset/'\nsamples = []\nwith tqdm(total=len(TRAIN)) as pbar:\n    for idx, row in TRAIN.iterrows():\n        pbar.update(1)\n        \n        if row.primary_label in most_represented_birds:\n            audio_file_path = os.path.join(input_dir, row.primary_label, row.filename)\n            samples += get_spectrograms(audio_file_path, row.primary_label, output_dir)\n            \nTRAIN_SPECS = shuffle(samples, random_state=RANDOM_SEED)\nprint('SUCCESSFULLY EXTRACTED {} SPECTROGRAMS'.format(len(TRAIN_SPECS)))","metadata":{"execution":{"iopub.status.busy":"2023-03-14T05:04:19.457906Z","iopub.execute_input":"2023-03-14T05:04:19.458398Z","iopub.status.idle":"2023-03-14T05:18:03.426864Z","shell.execute_reply.started":"2023-03-14T05:04:19.458361Z","shell.execute_reply":"2023-03-14T05:18:03.425300Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# plota os 12 primeiros espectrogramas de TRAIN_SPECS\nplt.figure(figsize=(15, 7))\nfor i in range(12):\n    spec = Image.open(TRAIN_SPECS[i])\n    plt.subplot(3, 4, i + 1)\n    plt.title(TRAIN_SPECS[i].split(os.sep)[-1])\n    plt.imshow(spec, origin='lower')","metadata":{"execution":{"iopub.status.busy":"2023-03-14T05:18:17.137428Z","iopub.execute_input":"2023-03-14T05:18:17.137983Z","iopub.status.idle":"2023-03-14T05:18:18.548626Z","shell.execute_reply.started":"2023-03-14T05:18:17.137928Z","shell.execute_reply":"2023-03-14T05:18:18.547631Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Perceba como alguns deles só tem uma fração de canto de pássaro. Isso é um problema que não vamos tratar nessa solução, por ser um trabalho inicial a AM. Vamos ignorar que talvez algumas amostras não contenham algum canto de pássaro.\n\n# 4. Subir as amostras de treinamento\n\nPor enquanto, nossos espectrogramas estão no diretório de trabalho. Se queremos treinar um modelo, nós temos que subi-los para a memória. Com centenas de milhares de espectrogramas extraídos, o banco de dados in-memory não é uma boa ideia. Mas nessa solução, carregar as amostras do disco e combinando eles em uma grande lista do NumPy é ok. É a forma mais simples de usar esses dados para treinar com Keras.\n\nO treinamento do nosso modelo vai ser feito usando uma forma simples da técnica de CNN (Convolutional Neural Network) para classificar as imagens de spectrogramas. E isso vai ser feito usando a API sequencial Keras para criar e treinar o modelo em algumas linhas de código.\n","metadata":{}},{"cell_type":"code","source":"# compila todas as amostras e adiciona os spectrogramas nos dados de treinamento, primary_labels nos dados de label\ntrain_specs, train_labels = [], []\nwith tqdm(total=len(TRAIN_SPECS)) as pbar:\n    for path in TRAIN_SPECS:\n        pbar.update(1)\n\n        # abrindo a imagem\n        spec = Image.open(path)\n\n        # converte em uma lista do numpy\n        spec = np.array(spec, dtype='float32')\n        \n        # normaliza entre 0.0 e 1.0 e exclui as amostras com NaN\n        spec -= spec.min()\n        spec /= spec.max()\n        if not spec.max() == 1.0 or not spec.min() == 0.0:\n            continue\n\n        # adiciona eixo para uma lista 2D\n        spec = np.expand_dims(spec, -1)\n\n        # adiciona nova dimensão para o tamanho do lote\n        spec = np.expand_dims(spec, 0)\n\n        # vstack serve para por exemplo:\n        # pixel-data com a altura (primeiro eixo), largura (segundo eixo), e os canais RGB (terceiro eixo)\n        # adiciona a amostra para treinar no modelo\n        if len(train_specs) == 0:\n            train_specs = spec\n        else:\n            train_specs = np.vstack((train_specs, spec))\n\n        # adiciona nos dados de label\n        target = np.zeros((len(LABELS)), dtype='float32')\n        bird = path.split(os.sep)[-2]\n        target[LABELS.index(bird)] = 1.0\n        if len(train_labels) == 0:\n            train_labels = target\n        else:\n            train_labels = np.vstack((train_labels, target))","metadata":{"execution":{"iopub.status.busy":"2023-03-14T05:38:10.878451Z","iopub.execute_input":"2023-03-14T05:38:10.879618Z","iopub.status.idle":"2023-03-14T06:51:22.873288Z","shell.execute_reply.started":"2023-03-14T05:38:10.879540Z","shell.execute_reply":"2023-03-14T06:51:22.872321Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 5. Construção de um modelo simples\n\n","metadata":{}},{"cell_type":"code","source":"f1_score=F1Score(num_classes=len(LABELS),average='macro',name='f1_score')","metadata":{"execution":{"iopub.status.busy":"2023-03-14T06:53:22.611822Z","iopub.execute_input":"2023-03-14T06:53:22.612271Z","iopub.status.idle":"2023-03-14T06:53:22.629115Z","shell.execute_reply.started":"2023-03-14T06:53:22.612228Z","shell.execute_reply":"2023-03-14T06:53:22.627379Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model 1","metadata":{}},{"cell_type":"code","source":"# tenha certeza que os experimentos sejam reproduzíveis\ntf.random.set_seed(RANDOM_SEED)\n\n# construa um modelo simples como a sequência de blocos de convolução\n# a cada bloco aumentou-se o número de filtros de output (2^i)\n# cada bloco tem a sequência CONV --> RELU --> BNORM --> MAXPOOL.\n# então, execute o agrupamento de média global e adicione 2 camadas densas\n# a ultima camada é nossa camada de classificação e o softmax é ativado\n# (é uma tarefa multi-label então sigmoid pode realmente ser a melhor escolha)\nmodel = tf.keras.Sequential([\n    \n    # primeiro bloco de convolução\n    tf.keras.layers.Conv2D(16, (3, 3), activation='relu', \n                           input_shape=(SPEC_SHAPE[0], SPEC_SHAPE[1], 1)),\n    tf.keras.layers.BatchNormalization(),\n    tf.keras.layers.MaxPooling2D((2, 2)),\n    \n    # segundo bloco de convolução\n    tf.keras.layers.Conv2D(32, (3, 3), activation='relu'),\n    tf.keras.layers.BatchNormalization(),\n    tf.keras.layers.MaxPooling2D((2, 2)), \n    \n    # terceiro bloco de convolução\n    tf.keras.layers.Conv2D(128, (3, 3), activation='relu'),\n    tf.keras.layers.BatchNormalization(),\n    tf.keras.layers.MaxPooling2D((2, 2)), \n    \n    # quarto bloco de convolução\n    tf.keras.layers.Conv2D(256, (3, 3), activation='relu'),\n    tf.keras.layers.BatchNormalization(),\n    tf.keras.layers.MaxPooling2D((2, 2)),\n        \n    \n    \n    # agrupamento global ao invés de função de nivelamento\n    tf.keras.layers.GlobalAveragePooling2D(), \n    \n    # bloco denso\n    tf.keras.layers.Dense(256, activation='relu'),  \n    tf.keras.layers.Dropout(0.5),\n#     tf.keras.layers.Dense(64, activation='relu'),   \n#     tf.keras.layers.Dropout(0.5),\n    \n    # camada de classificação\n    tf.keras.layers.Dense(len(LABELS), activation='softmax')\n])\n\n\n\nprint('MODEL HAS {} PARAMETERS.'.format(model.count_params()))","metadata":{"execution":{"iopub.status.busy":"2023-03-14T06:53:29.121772Z","iopub.execute_input":"2023-03-14T06:53:29.122192Z","iopub.status.idle":"2023-03-14T06:53:29.273863Z","shell.execute_reply.started":"2023-03-14T06:53:29.122141Z","shell.execute_reply":"2023-03-14T06:53:29.272715Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# compila o modelo e especifica otimizador, perda e métrica\nmodel.compile(optimizer=tf.keras.optimizers.Adam(lr=0.001),\n              loss=tf.keras.losses.CategoricalCrossentropy(label_smoothing=0.01),\n              metrics=['accuracy',f1_score])","metadata":{"execution":{"iopub.status.busy":"2023-03-14T06:53:34.980038Z","iopub.execute_input":"2023-03-14T06:53:34.980500Z","iopub.status.idle":"2023-03-14T06:53:34.994551Z","shell.execute_reply.started":"2023-03-14T06:53:34.980457Z","shell.execute_reply":"2023-03-14T06:53:34.993432Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# adiciona callbacks para reduzir a taxa de aprendizado se necessario,early stopping, e checkpoint saving\ncallbacks = [tf.keras.callbacks.ReduceLROnPlateau(monitor='val_f1_score', \n                                                  patience=2, \n                                                  verbose=1, \n                                                  mode='max',\n                                                  factor=0.5),\n             tf.keras.callbacks.EarlyStopping(monitor='val_loss', \n                                              verbose=1,\n                                              patience=20),\n             tf.keras.callbacks.ModelCheckpoint(filepath='best_model.h5', \n                                                monitor='val_f1_score',\n                                                mode='max',\n                                                verbose=1,\n                                                save_best_only=True)]","metadata":{"execution":{"iopub.status.busy":"2023-03-14T06:53:38.578592Z","iopub.execute_input":"2023-03-14T06:53:38.579068Z","iopub.status.idle":"2023-03-14T06:53:38.586555Z","shell.execute_reply.started":"2023-03-14T06:53:38.579029Z","shell.execute_reply":"2023-03-14T06:53:38.584920Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_history(history):\n    his=pd.DataFrame(history.history)\n    plt.subplots(1,2,figsize=(16,8))\n    \n    #loss:\n    plt.subplot(1,2,1)\n    plt.plot(range(len(his)),his['loss'],color='g',label='training')\n    plt.plot(range(len(his)),his['val_loss'],color='r',label='validation')\n    plt.legend()\n    plt.title('Loss')\n    \n    #accuracy\n    plt.subplot(1,2,2)\n    plt.plot(range(len(his)),his['accuracy'],color='g',label='training_acc')\n    plt.plot(range(len(his)),his['val_accuracy'],color='r',label='validation_acc')\n    \n#     #f1_score\n#     plt.plot(range(len(his)),his['f1_score'],color='steelblue',label='training_f1')\n#     plt.plot(range(len(his)),his['val_f1_score'],color='maroon',label='validation_f1')\n    \n    plt.legend()\n    plt.title('accuracy')\n    \n    plt.show()              ","metadata":{"execution":{"iopub.status.busy":"2023-03-14T06:53:47.986624Z","iopub.execute_input":"2023-03-14T06:53:47.987040Z","iopub.status.idle":"2023-03-14T06:53:47.996710Z","shell.execute_reply.started":"2023-03-14T06:53:47.987003Z","shell.execute_reply":"2023-03-14T06:53:47.995358Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# TRAINING","metadata":{}},{"cell_type":"code","source":"# vamos treinar o modelo para alguns formatos (epochs)\nhis=model.fit(train_specs, \n          train_labels,\n          batch_size=32,\n          validation_split=0.2,\n          callbacks=callbacks,\n          epochs=EPOCHS)\n\nplot_history(his)","metadata":{"execution":{"iopub.status.busy":"2023-03-14T06:53:53.601109Z","iopub.execute_input":"2023-03-14T06:53:53.601552Z","iopub.status.idle":"2023-03-14T07:50:48.543798Z","shell.execute_reply.started":"2023-03-14T06:53:53.601515Z","shell.execute_reply":"2023-03-14T07:50:48.542424Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model 2","metadata":{}},{"cell_type":"code","source":"tf.random.set_seed(RANDOM_SEED)\n\n# construa um modelo simples como a sequência de blocos de convolução\n# a cada bloco aumentou-se o número de filtros de output (2^i)\n# cada bloco tem a sequência CONV --> RELU --> BNORM --> MAXPOOL.\n# então, execute o agrupamento de média global e adicione 2 camadas densas\n# a ultima camada é nossa camada de classificação e o softmax é ativado\n# (é uma tarefa multi-label então sigmoid pode realmente ser a melhor escolha)\nmodel2 = tf.keras.Sequential([\n    \n    # primeiro bloco de convolução\n    tf.keras.layers.Conv2D(16, (3, 3), activation='relu', \n                           input_shape=(SPEC_SHAPE[0], SPEC_SHAPE[1], 1)),\n    tf.keras.layers.BatchNormalization(),\n    tf.keras.layers.MaxPooling2D((2, 2)),\n    \n    # segundo bloco de convolução\n    tf.keras.layers.Conv2D(32, (3, 3), activation='relu'),\n    tf.keras.layers.BatchNormalization(),\n    tf.keras.layers.MaxPooling2D((2, 2)), \n    \n    # terceiro bloco de convolução\n    tf.keras.layers.Conv2D(64, (3, 3), activation='relu'),\n    tf.keras.layers.BatchNormalization(),\n    tf.keras.layers.MaxPooling2D((2, 2)), \n    \n    # quarto bloco de convolução\n    tf.keras.layers.Conv2D(128, (3, 3), activation='relu'),\n    tf.keras.layers.BatchNormalization(),\n    tf.keras.layers.MaxPooling2D((2, 2)),\n        \n    \n    \n    # agrupamento global ao invés de função de nivelamento\n    tf.keras.layers.GlobalAveragePooling2D(), \n    \n    # bloco denso\n    tf.keras.layers.Dense(128, activation='relu'),  \n    tf.keras.layers.Dropout(0.5),\n#     tf.keras.layers.Dense(64, activation='relu'),   \n#     tf.keras.layers.Dropout(0.5),\n    \n    # camada de classificação\n    tf.keras.layers.Dense(len(LABELS), activation='softmax')\n])\n\n\n\nprint('MODEL HAS {} PARAMETERS.'.format(model2.count_params()))","metadata":{"execution":{"iopub.status.busy":"2023-03-14T07:50:48.694851Z","iopub.execute_input":"2023-03-14T07:50:48.695332Z","iopub.status.idle":"2023-03-14T07:50:48.831954Z","shell.execute_reply.started":"2023-03-14T07:50:48.695277Z","shell.execute_reply":"2023-03-14T07:50:48.830738Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# compila o modelo e especifica otimizador, perda e métrica\nmodel2.compile(optimizer=tf.keras.optimizers.Adam(lr=0.001),\n              loss=tf.keras.losses.CategoricalCrossentropy(label_smoothing=0.01),\n              metrics=['accuracy',f1_score])\n\n# adiciona callbacks para reduzir a taxa de aprendizado se necessario,early stopping, e checkpoint saving\ncallbacks = [tf.keras.callbacks.ReduceLROnPlateau(monitor='val_f1_score',\n                                                  mode='max',\n                                                  patience=2, \n                                                  verbose=1, \n                                                  factor=0.5),\n             tf.keras.callbacks.EarlyStopping(monitor='val_loss', \n                                              verbose=1,\n                                              patience=20),\n             tf.keras.callbacks.ModelCheckpoint(filepath='best_model2.h5', \n                                                mode='max',\n                                                monitor='val_f1_score',\n                                                verbose=0,\n                                                save_best_only=True)]","metadata":{"execution":{"iopub.status.busy":"2023-03-14T07:50:48.833875Z","iopub.execute_input":"2023-03-14T07:50:48.834364Z","iopub.status.idle":"2023-03-14T07:50:48.850825Z","shell.execute_reply.started":"2023-03-14T07:50:48.834311Z","shell.execute_reply":"2023-03-14T07:50:48.849775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# vamos treinar o modelo para alguns formatos (epochs)\nhis2=model2.fit(train_specs, \n          train_labels,\n          batch_size=32,\n          validation_split=0.2,\n          callbacks=callbacks,\n          verbose=1,\n          epochs=EPOCHS)\n\nplot_history(his2)","metadata":{"execution":{"iopub.status.busy":"2023-03-14T07:50:48.854353Z","iopub.execute_input":"2023-03-14T07:50:48.854864Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 6. Soundscape analysis","metadata":{}},{"cell_type":"markdown","source":"**Soundscape_analysis1**","metadata":{}},{"cell_type":"code","source":"# carrega o melhor checkpoint\nmodel = tf.keras.models.load_model('best_model.h5')\nmodel2= tf.keras.models.load_model('best_model2.h5')\n\n# escolher um soundscape\nsoundscape_path = '../input/birdclef-2021/train_soundscapes/28933_SSW_20170408.ogg'\n\n# abrir com librosa\nsig, rate = librosa.load(soundscape_path, sr=SAMPLE_RATE)\n\n# armazenar resultados para podermos analisar depois\ndata = {'row_id': [], 'prediction': [], 'score': []}\n\n# dividir o sinal em pedaços de 5s\n# como fizemos anteriormente\nsig_splits = []\nfor i in range(0, len(sig), int(SIGNAL_LENGTH * SAMPLE_RATE)):\n    split = sig[i:i + int(SIGNAL_LENGTH * SAMPLE_RATE)]\n\n    # End of signal?\n    if len(split) < int(SIGNAL_LENGTH * SAMPLE_RATE):\n        break\n\n    sig_splits.append(split)\n    \n\n# pegar o espectrograma e rodar inferência em cada um deles\n# esse é o mesmo processo como foi feito\n# gere amostras de treinamento\nseconds, scnt = 0, 0\nfor chunk in sig_splits:\n    \n    # mantenha registro do final de cada pedaço\n    seconds += 5\n        \n    # pegue o espectrograma\n    hop_length = int(SIGNAL_LENGTH * SAMPLE_RATE / (SPEC_SHAPE[1] - 1))\n    mel_spec = librosa.feature.melspectrogram(y=chunk, \n                                              sr=SAMPLE_RATE, \n                                              n_fft=1024, \n                                              hop_length=hop_length, \n                                              n_mels=SPEC_SHAPE[0], \n                                              fmin=FMIN, \n                                              fmax=FMAX)\n\n    mel_spec = librosa.power_to_db(mel_spec, ref=np.max) \n\n    # normalize para bater com a faixa de valor usada durante o treinamento\n    mel_spec -= mel_spec.min()\n    mel_spec /= mel_spec.max()\n    \n    # adiciona eixo para lista 2D\n    mel_spec = np.expand_dims(mel_spec, -1)\n\n    # adiciona nova dimensão para o tamanho do lote\n    mel_spec = np.expand_dims(mel_spec, 0)\n    \n    # predição\n    p = model.predict(mel_spec)[0] + model2.predict(mel_spec)[0] \n    \n    \n    # pegar as espécies com maiores notas\n    idx = p.argmax()\n    species = LABELS[idx]\n    score = p[idx]\n    \n    # prepare a entrada da submissão\n    data['row_id'].append(soundscape_path.split(os.sep)[-1].rsplit('_', 1)[0] + \n                          '_' + str(seconds))    \n    \n    # decidir quais são espécies ou \"nocall\"s aplicando um filtro\n    if score > 0.4:\n        data['prediction'].append(species)\n        scnt += 1\n    else:\n        data['prediction'].append('nocall')\n        \n    # adicione as notas\n    data['score'].append(score)\n        \nprint('SOUNSCAPE ANALYSIS DONE. FOUND {} BIRDS.'.format(scnt))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# construa um novo data frame\nresults = pd.DataFrame(data, columns = ['row_id', 'prediction', 'score'])\n\n# mescle com a fonte para poder inspecionar\ngt = pd.read_csv('../input/birdclef-2021/train_soundscape_labels.csv',)\nresults = pd.merge(gt, results, on='row_id')\n\n# vamos ver as primeiras 50 entradas\nresults.head(50)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Soundscape_analysis2_2**","metadata":{}},{"cell_type":"code","source":"\n# Pick a soundscape\nsoundscape_path = '../input/birdclef-2021/train_soundscapes/7843_SSW_20170325.ogg'\n\n# Open it with librosa\nsig, rate = librosa.load(soundscape_path, sr=SAMPLE_RATE)\n\n# Store results so that we can analyze them later\ndata = {'row_id': [], 'prediction': [], 'score': []}\n\n# Split signal into 5-second chunks\n# Just like we did before (well, this could actually be a seperate function)\nsig_splits = []\nfor i in range(0, len(sig), int(SIGNAL_LENGTH * SAMPLE_RATE)):\n    split = sig[i:i + int(SIGNAL_LENGTH * SAMPLE_RATE)]\n\n    # End of signal?\n    if len(split) < int(SIGNAL_LENGTH * SAMPLE_RATE):\n        break\n\n    sig_splits.append(split)\n    \n# Get the spectrograms and run inference on each of them\n# This should be the exact same process as we used to\n# generate training samples!\nseconds, scnt = 0, 0\nfor chunk in sig_splits:\n    \n    # Keep track of the end time of each chunk\n    seconds += 5\n        \n    # Get the spectrogram\n    hop_length = int(SIGNAL_LENGTH * SAMPLE_RATE / (SPEC_SHAPE[1] - 1))\n    mel_spec = librosa.feature.melspectrogram(y=chunk, \n                                              sr=SAMPLE_RATE, \n                                              n_fft=1024, \n                                              hop_length=hop_length, \n                                              n_mels=SPEC_SHAPE[0], \n                                              fmin=FMIN, \n                                              fmax=FMAX)\n\n    mel_spec = librosa.power_to_db(mel_spec, ref=np.max) \n\n    # Normalize to match the value range we used during training.\n    # That's something you should always double check!\n    mel_spec -= mel_spec.min()\n    mel_spec /= mel_spec.max()\n    \n    # Add channel axis to 2D array\n    mel_spec = np.expand_dims(mel_spec, -1)\n\n    # Add new dimension for batch size\n    mel_spec = np.expand_dims(mel_spec, 0)\n    \n    # Predict\n    p = 0.5*model.predict(mel_spec)[0] + 0.5*model2.predict(mel_spec)[0]\n    \n    \n    # Get highest scoring species\n    idx = p.argmax()\n    species = LABELS[idx]\n    score = p[idx]\n    \n    # Prepare submission entry\n    data['row_id'].append(soundscape_path.split(os.sep)[-1].rsplit('_', 1)[0] + \n                          '_' + str(seconds))    \n    \n    # Decide if it's a \"nocall\" or a species by applying a threshold\n    if score > 0.45:\n        data['prediction'].append(species)\n        scnt += 1\n    else:\n        data['prediction'].append('nocall')\n        \n    # Add the confidence score as well\n    data['score'].append(score)\n        \nprint('SOUNSCAPE ANALYSIS DONE. FOUND {} BIRDS.'.format(scnt))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Make a new data frame\nresults = pd.DataFrame(data, columns = ['row_id', 'prediction', 'score'])\n\n# Merge with ground truth so we can inspect\ngt = pd.read_csv('../input/birdclef-2021/train_soundscape_labels.csv',)\nresults = pd.merge(gt, results, on='row_id')\n\n# Let's look at the first 50 entries\nresults.head(50)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Soundscape_analysis_3**","metadata":{}},{"cell_type":"code","source":"# Pick a soundscape\nsoundscape_path = '../input/birdclef-2021/train_soundscapes/26709_SSW_20170701.ogg'\n\n# Open it with librosa\nsig, rate = librosa.load(soundscape_path, sr=SAMPLE_RATE)\n\n# Store results so that we can analyze them later\ndata = {'row_id': [], 'prediction': [], 'score': []}\n\n# Split signal into 5-second chunks\n# Just like we did before (well, this could actually be a seperate function)\nsig_splits = []\nfor i in range(0, len(sig), int(SIGNAL_LENGTH * SAMPLE_RATE)):\n    split = sig[i:i + int(SIGNAL_LENGTH * SAMPLE_RATE)]\n\n    # End of signal?\n    if len(split) < int(SIGNAL_LENGTH * SAMPLE_RATE):\n        break\n\n    sig_splits.append(split)\n    \n# Get the spectrograms and run inference on each of them\n# This should be the exact same process as we used to\n# generate training samples!\nseconds, scnt = 0, 0\nfor chunk in sig_splits:\n    \n    # Keep track of the end time of each chunk\n    seconds += 5\n        \n    # Get the spectrogram\n    hop_length = int(SIGNAL_LENGTH * SAMPLE_RATE / (SPEC_SHAPE[1] - 1))\n    mel_spec = librosa.feature.melspectrogram(y=chunk, \n                                              sr=SAMPLE_RATE, \n                                              n_fft=1024, \n                                              hop_length=hop_length, \n                                              n_mels=SPEC_SHAPE[0], \n                                              fmin=FMIN, \n                                              fmax=FMAX)\n\n    mel_spec = librosa.power_to_db(mel_spec, ref=np.max) \n\n    # Normalize to match the value range we used during training.\n    # That's something you should always double check!\n    mel_spec -= mel_spec.min()\n    mel_spec /= mel_spec.max()\n    \n    # Add channel axis to 2D array\n    mel_spec = np.expand_dims(mel_spec, -1)\n\n    # Add new dimension for batch size\n    mel_spec = np.expand_dims(mel_spec, 0)\n    \n    # Predict\n    p = 0.5*model.predict(mel_spec)[0] + 0.5*model2.predict(mel_spec)[0]\n    \n    \n    # Get highest scoring species\n    idx = p.argmax()\n    species = LABELS[idx]\n    score = p[idx]\n    \n    # Prepare submission entry\n    data['row_id'].append(soundscape_path.split(os.sep)[-1].rsplit('_', 1)[0] + \n                          '_' + str(seconds))    \n    \n    # Decide if it's a \"nocall\" or a species by applying a threshold\n    if score > 0.3:\n        data['prediction'].append(species)\n        scnt += 1\n    else:\n        data['prediction'].append('nocall')\n        \n    # Add the confidence score as well\n    data['score'].append(score)\n        \nprint('SOUNSCAPE ANALYSIS DONE. FOUND {} BIRDS.'.format(scnt))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Make a new data frame\nresults = pd.DataFrame(data, columns = ['row_id', 'prediction', 'score'])\n\n# Merge with ground truth so we can inspect\ngt = pd.read_csv('../input/birdclef-2021/train_soundscape_labels.csv',)\nresults = pd.merge(gt, results, on='row_id')\n\n# Let's look at the first 50 entries\nresults.head(50)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Prediction\n**Inference notebook** [ https://www.kaggle.com/virajkadam/birdclef-inference ]","metadata":{}}]}