{"cells":[{"metadata":{},"cell_type":"markdown","source":" Neste notebook implementei um algoritmo simples de segmentação utilizado em [Frog classification using machine learning techniques](https://www.sciencedirect.com/science/article/abs/pii/S0957417408001504). É um algoritmo simples mas que pode ser considerado uma boa heuristica para evitar selecionar partes sem cantos.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Importando Libs","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport librosa\nimport matplotlib.pyplot as plt\nimport os\nimport cv2\nimport IPython.display as ipd\nfrom IPython.display import Audio, IFrame, display\nimport plotly.graph_objects as go\nimport librosa\nimport librosa.display\n\n\nimport warnings\nwarnings.filterwarnings('ignore')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# O algoritmo\nO algoritmo de segmentação basicamente consiste na ideia de selecionar e retirar a parte do audio que possui a maior amplitude. Depois fazer a mesma coisa para o restante do audio, e repetir até que o audio restante não ultrapasse uma amplitude pre-determinada.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# The Function return the segments extracted from audio and the rest of audio\ndef segmentation(x,tr):\n    \n    resp = [];\n    \n    while(len(x) >= (sr*2)):\n        if(max(x) < tr):\n            break\n            \n        time_amplitude_max = np.argmax(x)\n        \n        #Higher amplitude is before 1s \n        if(time_amplitude_max < ((1)*sr)):\n            resp.append(x[:2*sr])\n            x = x[2*sr:]\n            \n        #Higher amplitude is on the last 1s     \n        elif(time_amplitude_max > (len(x) - ((1)*sr))):\n            resp.append(x[-(2*sr):])\n            x = x[:-2*sr]\n            \n        else:\n            resp.append(x[time_amplitude_max-int((1)*sr):time_amplitude_max+int((1)*sr)])\n            x = np.concatenate((x[:time_amplitude_max-int((1)*sr)-1],x[time_amplitude_max+int((1)*sr)+1:]))\n            \n    return resp,x     ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Exemplo 1\nEsse primeiro exemplo é muito bom pois temos um audio que é meio dividido entre partes de silencio e canto, esse passaro foi legal.","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"x , sr = librosa.load(\"../input/birdsong-recognition/train_audio/cacwre/XC132895.mp3\")\nlibrosa.display.waveplot(x, sr=sr)\nAudio(x, rate=sr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Aqui aplicamos o algoritmo ao aúdio, com o threshold = 0.25","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"segments,resto = segmentation(x,0.25)\nprint(\"Foram recuperados {} Segmentos\".format(len(segments)));","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Em baixo temos os pedaço do audio que não foi selecionado","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"librosa.display.waveplot(resto, sr=sr)\nAudio(resto, rate=sr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"O segmento com maior amplitude.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"librosa.display.waveplot(segments[1], sr=sr)\nAudio(segments[1], rate=sr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"melspec = librosa.feature.melspectrogram(segments[1],sr=sr)\nmelspec = librosa.power_to_db(melspec).astype(np.float32)\nplt.figure(figsize=(10, 4))\nlibrosa.display.specshow(melspec, x_axis='time',y_axis='mel', sr=sr)\nplt.colorbar(format='%+2.0f dB')\nplt.title('Mel-Spectrogram')\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Exemplo 2","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"x , sr = librosa.load(\"../input/birdsong-recognition/train_audio/btywar/XC133590.mp3\")\nlibrosa.display.waveplot(x, sr=sr)\nAudio(x, rate=sr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Aqui foi recuperado somente 1 Segmento(Sem canto), mesmo havendo outras partes contendo um canto","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"segments,resto = segmentation(x,0.25)\nprint(\"Foram recuperados {} Segmentos\".format(len(segments)));","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"librosa.display.waveplot(segments[0], sr=sr)\nAudio(segments[0], rate=sr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"melspec = librosa.feature.melspectrogram(segments[0],sr=sr)\nmelspec = librosa.power_to_db(melspec).astype(np.float32)\nplt.figure(figsize=(10, 4))\nlibrosa.display.specshow(melspec, x_axis='time',y_axis='mel', sr=sr)\nplt.colorbar(format='%+2.0f dB')\nplt.title('Mel-Spectrogram')\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Exemplo 3","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"x , sr = librosa.load(\"../input/birdsong-recognition/train_audio/killde/XC143721.mp3\")\nlibrosa.display.waveplot(x, sr=sr)\nAudio(x, rate=sr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"segments,resto = segmentation(x,0.25)\nprint(\"Foram recuperados {} Segmentos\".format(len(segments)));","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"librosa.display.waveplot(segments[0], sr=sr)\nAudio(segments[0], rate=sr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"melspec = librosa.feature.melspectrogram(segments[0],sr=sr)\nmelspec = librosa.power_to_db(melspec).astype(np.float32)\nplt.figure(figsize=(10, 4))\nlibrosa.display.specshow(melspec, x_axis='time',y_axis='mel', sr=sr)\nplt.colorbar(format='%+2.0f dB')\nplt.title('Mel-Spectrogram')\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Vamos testar os exemplos acima normalizando antes da segmentação","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def normaliza(x):\n    mi = x.mean()\n    sigma = np.std(x)\n    \n    x = x-mi;\n    return (x/sigma) ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Exemplo 1","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"x , sr = librosa.load(\"../input/birdsong-recognition/train_audio/cacwre/XC132895.mp3\")\nx = normaliza(x)               \nlibrosa.display.waveplot(x, sr=sr)\nAudio(x, rate=sr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Aparentemente para threshold = 1, são selecionados muitos segmentos.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"segments,resto = segmentation(x,1)\nprint(\"Foram recuperados {} Segmentos\".format(len(segments)));","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Alguns sem canto.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"librosa.display.waveplot(segments[51], sr=sr)\nAudio(segments[51], rate=sr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Achei 5 melhor","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"segments,resto = segmentation(x,5)\nprint(\"Foram recuperados {} Segmentos\".format(len(segments)));","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"librosa.display.waveplot(segments[21], sr=sr)\nAudio(segments[21], rate=sr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"melspec = librosa.feature.melspectrogram(segments[0],sr=sr)\nmelspec = librosa.power_to_db(melspec).astype(np.float32)\nplt.figure(figsize=(10, 4))\nlibrosa.display.specshow(melspec, x_axis='time',y_axis='mel', sr=sr)\nplt.colorbar(format='%+2.0f dB')\nplt.title('Mel-Spectrogram')\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Exemplo 2","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Esse caso 2 é interessante, porque ele ta meio baixo, e tem um momento que tem um ruido perto do microfone. Normalizando a gente consegue selecionar mais cantos, apesar ainda de vir algumas coisas como uma buzina.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"x , sr = librosa.load(\"../input/birdsong-recognition/train_audio/btywar/XC133590.mp3\")\nx = normaliza(x)\nlibrosa.display.waveplot(x, sr=sr)\nAudio(x, rate=sr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"segments,resto = segmentation(x,5)\nprint(\"Foram recuperados {} Segmentos\".format(len(segments)));","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"O primeiro segmento escolhido não tem nada.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"librosa.display.waveplot(segments[0], sr=sr)\nAudio(segments[0], rate=sr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"melspec = librosa.feature.melspectrogram(segments[0],sr=sr)\nmelspec = librosa.power_to_db(melspec).astype(np.float32)\nplt.figure(figsize=(10, 4))\nlibrosa.display.specshow(melspec, x_axis='time',y_axis='mel', sr=sr)\nplt.colorbar(format='%+2.0f dB')\nplt.title('Mel-Spectrogram')\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"librosa.display.waveplot(segments[2], sr=sr)\nAudio(segments[2], rate=sr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"No entanto nestes dois audios/spectrogramas abaixo a gente consegue pegar perfeitamente o canto.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"melspec = librosa.feature.melspectrogram(segments[2],sr=sr)\nmelspec = librosa.power_to_db(melspec).astype(np.float32)\nplt.figure(figsize=(10, 4))\nlibrosa.display.specshow(melspec, x_axis='time',y_axis='mel', sr=sr)\nplt.colorbar(format='%+2.0f dB')\nplt.title('Mel-Spectrogram')\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"librosa.display.waveplot(segments[4], sr=sr)\nAudio(segments[4], rate=sr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"melspec = librosa.feature.melspectrogram(segments[4],sr=sr)\nmelspec = librosa.power_to_db(melspec).astype(np.float32)\nplt.figure(figsize=(10, 4))\nlibrosa.display.specshow(melspec, x_axis='time',y_axis='mel', sr=sr)\nplt.colorbar(format='%+2.0f dB')\nplt.title('Mel-Spectrogram')\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Exemplo 3","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Esse é mais Trivial","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"x , sr = librosa.load(\"../input/birdsong-recognition/train_audio/killde/XC143721.mp3\")\nx = normaliza(x)\nlibrosa.display.waveplot(x, sr=sr)\nAudio(x, rate=sr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"segments,resto = segmentation(x,5)\nprint(\"Foram recuperados {} Segmentos\".format(len(segments)));","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"librosa.display.waveplot(segments[0], sr=sr)\nAudio(segments[0], rate=sr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"melspec = librosa.feature.melspectrogram(segments[0],sr=sr)\nmelspec = librosa.power_to_db(melspec).astype(np.float32)\nplt.figure(figsize=(10, 4))\nlibrosa.display.specshow(melspec, x_axis='time',y_axis='mel', sr=sr)\nplt.colorbar(format='%+2.0f dB')\nplt.title('Mel-Spectrogram')\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"librosa.display.waveplot(segments[1], sr=sr)\nAudio(segments[1], rate=sr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"melspec = librosa.feature.melspectrogram(segments[1],sr=sr)\nmelspec = librosa.power_to_db(melspec).astype(np.float32)\nplt.figure(figsize=(10, 4))\nlibrosa.display.specshow(melspec, x_axis='time',y_axis='mel', sr=sr)\nplt.colorbar(format='%+2.0f dB')\nplt.title('Mel-Spectrogram')\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}