{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# BirdCLEF2022\n\n## 比赛简介\n\n这个比赛主要任务是对音频中的鸟叫声进行识别，测试时会给定音频要求判断给定时间段中是否有某种特定鸟类的叫声。\n\n请认真阅读比赛要求指南：[比赛首页](https://www.kaggle.com/c/birdclef-2022)\n\n## 关于本notebook\n\n本notebook会对所给数据集进行基本分析及预处理，将音频文件进行转换，做简单的特征工程操作。\n\n## 进度\n\n - [x] 数据集文件总览\n - [x] 音频样例分析\n - [x] 音频转换\n - [x] training set预处理\n \n## 一些参考的文章及notebook\n\n[https://www.kaggle.com/jirkaborovec/birdclef-eda-baseline-flash-efficientnet/data](https://www.kaggle.com/jirkaborovec/birdclef-eda-baseline-flash-efficientnet/data)\n\n[https://www.kaggle.com/hasanbasriakcay/birdclef22-eda-noise-reduction](https://www.kaggle.com/hasanbasriakcay/birdclef22-eda-noise-reduction)\n\n[https://www.kaggle.com/drcapa/birdclef-2022-starter](https://www.kaggle.com/drcapa/birdclef-2022-starter)\n\n感谢这些作者做出的贡献","metadata":{}},{"cell_type":"markdown","source":"# Import libraries","metadata":{}},{"cell_type":"code","source":"# base\nimport os\nimport json\nimport random\nimport pandas as pd\nimport numpy as np\nimport torch\n\n# plot\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom PIL import Image\n\n# For exploring audio files\nimport librosa\nimport librosa.display\nimport torchaudio\nimport IPython.display as ipd\n\n# tqdm\nfrom tqdm import tqdm\n\nimport warnings\nwarnings.filterwarnings('ignore')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 数据总览","metadata":{}},{"cell_type":"markdown","source":"## 所有文件","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"base_dir = '../input/birdclef-2022/'\nos.listdir(base_dir)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1.test_soundscapes文件\n\n里面存放要识别的soundscape文件，下面是可下载数据集的样例","metadata":{}},{"cell_type":"code","source":"ipd.Audio('../input/birdclef-2022/test_soundscapes/soundscape_453028782.ogg')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 2.train_audio\n\n里面存放用于训练的声音，文件夹名就是鸟名","metadata":{}},{"cell_type":"code","source":"train_audio_dir = base_dir + 'train_audio/'\nprint(train_audio_dir + '#')\nprint(os.listdir(train_audio_dir)[:5])\nprint()\nprint(train_audio_dir+'afrsil1'+'#')\nprint(os.listdir(train_audio_dir+'afrsil1'))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# afrsil1的一个声音展示\nipd.Audio('../input/birdclef-2022/train_audio/afrsil1/XC125458.ogg')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 3. eBird_Txonomy_v2021.csv\n\n这个csv展示的时不同物种间的关系表","metadata":{}},{"cell_type":"code","source":"ebird_df = pd.read_csv('../input/birdclef-2022/eBird_Taxonomy_v2021.csv')\nebird_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 4.sample_submission.csv\n\n提交的样例csv。从该文件可以看出提交格式。\n\n - row_id: test.csv中的对应主键\n - target: 判断是否有鸟的声音，true/false\n\n**注意：**可下载的数据集是不全的，同样sample_submission.csv文件也是不全的，但当用private数据集测试时sample_submission.csv大小是对的，所以可以通过修改sample来提交正确格式的submission，这个可以通过将sample中的target改为全true进行验证，能得到分数0.51","metadata":{}},{"cell_type":"code","source":"sample_sub_df = pd.read_csv('../input/birdclef-2022/sample_submission.csv')\nsample_sub_df","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 5.scored_birds.json\n\n一个重要的json文件，里面记录了21种鸟类的名字，这些鸟是需要判断是否出现在测试音频中的鸟类，所以生成submission可以通过遍历scored birds生成true/false而不用遍历所有的test.csv，这可以减少inference过程中load data的时间。","metadata":{}},{"cell_type":"code","source":"with open(base_dir+'scored_birds.json') as f:\n    scored_birds = json.load(f)\n\n    \nprint('scored birds:')\nprint(scored_birds[:5])\nprint()\nprint('length:')\nprint(len(scored_birds))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 6.test.csv\n\n测试集的元数据。只有前三行可供下载;完整的test.csv在隐藏的测试集中提供。\n\n - row_id:表主键\n - file_id:该行对应音频\n - bird:要判断的鸟类，只有scored birds中的鸟类，并且判断是每5秒判断一次\n - end_time:该音频窗口的结束时间(5的倍数）","metadata":{}},{"cell_type":"code","source":"test_df = pd.read_csv('../input/birdclef-2022/test.csv')\ntest_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 7.train_metadata.csv\n\n所有的元数据。主要有用的数据列如下：\n\n - primary_label：鸟的名字\n - secondary_labels：由录音师注释的background species\n - author: 提供录音的作者\n - filename: 文件路径\n - rating: 标志的可信度，1最低5最高，0代表目前还没有排名","metadata":{}},{"cell_type":"code","source":"train_metadata_df = pd.read_csv('../input/birdclef-2022/train_metadata.csv')\ntrain_metadata_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 音频处理样例\n\n下面讲述如何处理音频文件，让其能应用到神经网络及其他类似算法中。主要使用torchaudio库，当然也可以使用librosa库进行音频处理","metadata":{}},{"cell_type":"markdown","source":"## import\n\n导入一些必要的包...\n","metadata":{}},{"cell_type":"code","source":"!pip install -q noisereduce","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch\nimport torchaudio\nimport noisereduce as nr","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 音频读取\n\n通过torchaudio.load读取音频，返回signal和sample rate。由输出可以看出音频是单通道并且length为1920000=60×32k（sample时长为1min）。","metadata":{}},{"cell_type":"code","source":"sample_soundscape_dir = '../input/birdclef-2022/test_soundscapes/soundscape_453028782.ogg'\nsam_sig, sam_sr = torchaudio.load(sample_soundscape_dir)\nprint('sigal:\\n',sam_sig)\nprint()\nprint('shape:\\n',sam_sig.shape)\nprint()\nprint('sample rate:\\n',sam_sr)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 可视化：","metadata":{}},{"cell_type":"code","source":"# y = sam_sig[0]\n# x = np.arange(sam_sig.shape[1])\n# plt.plot(x,y)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 转换成双通道\n\n一些声音文件是单声道（即1个音频通道），而大多数则是立体声（即2个音频通道）。由于我们的模型期望所有项目都具有相同的尺寸，因此我们将第一个通道复制到第二个通道，从而将单声道文件转换为立体声。","metadata":{}},{"cell_type":"code","source":"  # ----------------------------\n  # Convert the given audio to the desired number of channels\n  # ----------------------------\n  @staticmethod\n  def rechannel(aud, new_channel):\n    sig, sr = aud\n\n    if (sig.shape[0] == new_channel):\n      # Nothing to do\n      return aud\n\n    if (new_channel == 1):\n      # Convert from stereo to mono by selecting only the first channel\n      resig = sig[:1, :]\n    else:\n      # Convert from mono to stereo by duplicating the first channel\n      resig = torch.cat([sig, sig])\n\n    return ((resig, sr))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 梅尔谱图\n\n一般会将音频转为梅尔普图作为输入加入到深度学习模型中，可以通过如下方法，默认进行了去噪。参考了[https://www.kaggle.com/jirkaborovec/birdclef-eda-baseline-flash-efficientnet/notebook](https://www.kaggle.com/jirkaborovec/birdclef-eda-baseline-flash-efficientnet/notebook)","metadata":{}},{"cell_type":"code","source":"def create_spectrogram(fname, reduce_noise: bool = False, max_length = int(1e7), device = \"cpu\"):\n    waveform, sample_rate = torchaudio.load(fname)\n    waveform = waveform[0][:max_length]\n    transform = torchaudio.transforms.Spectrogram(n_fft=1800, win_length=1024).to(device)\n    if reduce_noise:\n        waveform = torch.tensor(nr.reduce_noise(y=waveform, sr=sample_rate, win_length=transform.win_length, use_tqdm=False, n_jobs=-1))\n    spectrogram = transform(waveform.to(device))\n    return torch.log(spectrogram).numpy()\n\nprint(sample_soundscape_dir+'#')\nsg = create_spectrogram(sample_soundscape_dir, reduce_noise=True)\nprint(sg)\nprint()\nprint('shape:', sg.shape)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.imshow(sg)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 数据预处理\n\n将所有的音频文件转换为img，需要对上述代码进行重构。将音频裁剪成5s长度，与测试集对齐。注意是否降噪对结果有巨大影响，同时图片的点很多负无穷，因为数值问题，没办法进行很好的归一化，保存时直接截断保存即可\n","metadata":{}},{"cell_type":"code","source":"from torch.utils.data import DataLoader\nfrom math import ceil\n\nSIGNAL_LENGTH = 5 # 5s\nSAMPLE_RATE = 32000 # 32kHz\n\ndef get_spectrograms(fname, \n                     reduce_noise: bool = False,\n                     sample_rate: int = 32000,\n                     frame_length: int = 5,\n                     device = \"cpu\"):\n    waveform, sample_rate = torchaudio.load(fname)\n    transform = torchaudio.transforms.Spectrogram(n_fft=1800, win_length=512).to(device)\n    if reduce_noise:\n        waveform = torch.tensor(nr.reduce_noise(\n            y=waveform, sr=sample_rate, win_length=transform.win_length, use_tqdm=True, n_jobs=2,\n        ))\n    sig = waveform[0]\n    \n    # Split signal into five second chunks\n    \n    sig_splits = []\n    spectrograms = []\n#     print(waveform)\n#     print(sig)\n#     print(len(sig))\n#     print(waveform.size()[-1])\n#     print(sig[:int(frame_length * sample_rate)])\n#     print(waveform[0][:int(frame_length * sample_rate)])\n    \n    for i in range(0, len(sig), int(frame_length * sample_rate)):\n        split = sig[i:i + int(frame_length * sample_rate)]\n \n        # End of signal?\n        if len(split) < int(frame_length * sample_rate):\n            break\n        \n        frame = split\n#         sg = torchaudio.transforms.AmplitudeToDB(top_db=80)(waveform[:][i:i + int(frame_length * sample_rate)])\n#         np.array(sg)\n \n        db = torch.log(transform(frame.to(device)))\n#         db[db<-80]=-80\n        \n        sg = np.nan_to_num(db.cpu().numpy())\n#         sg -= sg.min()\n#         sg /= sg.max()\n        spectrograms.append(sg)\n    \n    return spectrograms\n\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"下面展示不去噪的结果图片","metadata":{}},{"cell_type":"code","source":"# sample for test\nsgs = get_spectrograms(sample_soundscape_dir, reduce_noise=False)\n\nfig, axarr = plt.subplots(ncols=len(sgs), figsize=(4 * len(sgs), 4))\nfor i, sg in enumerate(sgs):\n    ax = axarr[i].imshow(sg, vmin=-80, vmax=20)\nplt.colorbar(ax)\nfig.tight_layout()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"上述结果是未去噪的结果图像，下面将展示去噪后的图像。可以看出去噪对梅尔普图影响显著。","metadata":{}},{"cell_type":"code","source":"sgs = get_spectrograms(sample_soundscape_dir, reduce_noise=True)\n\nfig, axarr = plt.subplots(ncols=len(sgs), figsize=(4 * len(sgs), 4))\nfor i, sg in enumerate(sgs):\n    ax = axarr[i].imshow(sg, vmin=-80, vmax=20)\nplt.colorbar(ax)\nfig.tight_layout()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"接下来只需要将其运用到所有training set中即可。","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}