{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Mel-Frequency Ceptral Coeffienents(MFCC) feature extraction for Sound Classification\n\nFor sound classification like the [Cornell Birdcall Identification](https://www.kaggle.com/c/birdsong-recognition/overview) is usually using the MFCC feature.\n\nIt takes few hours for Cornell Birdcall Identification datasets. I will share extracted feature as dataset after the execution in colab.\n\nIn this notebook, I just use 3 mp3 files for each bird class. (check the LIMIT variable)\n\nPlease enjoy it and don't forget to vote it. \n\nFeel free to give an advice.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Mel-Frequency Cepstral Coefficients (MFCCs)\n\n![](https://img1.daumcdn.net/thumb/R1280x0/?scode=mtistory2&fname=https%3A%2F%2Fk.kakaocdn.net%2Fdn%2FTsu71%2FbtqETBgoxsP%2F7rgu73Uyc3isPddR9q1ZOK%2Fimg.png)\n\nThe log-spectrum already takes into account perceptual sensitivity on the magnitude axis, by expressing magnitudes on the logarithmic-axis. The other dimension is then the frequency axis. \n\nThere exists a multitude of different criteria with which to quantify accuracy on the frequency scale and there are, correspondingly, a multitude of perceptually motivated frequency scales including the equivalent rectangular bandwidth (ERB) scale, the Bark scale, and the mel-scale. Probably through an abritrary choice mainly due to tradition, in this context we will focus on the mel-scale. This scale describes the perceptual distance between pitches of different frequencies. \n\nThough the argumentation for the MFCCs is not without problems, it has become the most used feature in speech and audio recognition applications. It is used because it works and because it has relatively low complexity and it is straightforward to implement. Simply stated,\n\nif you're unsure which inputs to give to a speech and audio recognition engine, try first the MFCCs.\n\n![](https://img1.daumcdn.net/thumb/R1280x0/?scode=mtistory2&fname=https%3A%2F%2Fk.kakaocdn.net%2Fdn%2FlR19O%2FbtqETBgoAUx%2F8mcBOUb3mJkHW92sGyGMB0%2Fimg.png)\n\nThe beneficial properties of the MFCCs include:\n\nQuantifies the gross-shape of the spectrum (the spectral envelope), which is important in, for example, identification of vowels. At the same time, it removes fine spectral structure (micro-level structure), which is often less important. It thus focuses on that part of the signal which is typically most informative.\nStraightforward and computationally reasonably efficient calculation.\nTheir performance is well-tested and -understood.\nSome of the issues with the MFCC include:\n\nThe choice of perceptual scale is not well-motivated. Scales such as the ERB or gamma-tone filterbanks might be better suited. However, these alternative filterbanks have not demonstrated consistent benefit, whereby the mel-scale has persisted.\nMFCCs are not robust to noise. That is, the performance of MFCCs in presence of additive noise, in comparison to other features, has not always been good. \nThe choice of triangular weighting filters wk,h is arbitrary and not based on well-grounded motivations. Alternatives have been presented, but they have not gained popularity, probably due to minor effect on outcome.\nThe MFCCs work well in analysis but for synthesis, they are problematic. Namely, it is difficult to find an inverse transform (from MFCCs to power spectra) which is simultaneously unbiased (=accurate) and congruent with its physical representation (=power spectrum must be positive).\n\nref: https://wiki.aalto.fi/display/ITSP/Cepstrum+and+MFCC <br/>\nref: https://melon1024.github.io/ssc/","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\nimport os\nimport glob\nimport librosa\nimport librosa.display\nfrom tqdm import tqdm_notebook as tqdm\nfrom keras.models import Model\nfrom keras.utils import np_utils\n\nimport warnings\nwarnings.filterwarnings('ignore')\n\n%matplotlib inline \nimport matplotlib.pyplot as plt","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"LIMIT = 3","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"!ls ../input/birdsong-recognition","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train = pd.read_csv('../input/birdsong-recognition/train.csv')\ndf_train","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls ../input/birdsong-recognition/train_audio\n\ntrain_dir = '../input/birdsong-recognition/train_audio'\ntest_idr = '../input/birdsong-recognition/test_audio'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Extract Feature using MFCC()","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def mfcc_extract(filename):\n    try:\n        y, sr  = librosa.load(filename, sr = 44100)\n        mfcc = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=13, n_fft=int(0.02*sr),hop_length=int(0.01*sr))\n        return mfcc\n    except:\n        return","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def parse_audio_files(parent_dir, sub_dirs, limit):\n    labels = []\n    features = []\n    for label, sub_dir in enumerate(tqdm(sub_dirs)):\n        i = 0\n        for fn in glob.glob(os.path.join(parent_dir,sub_dir,\"*.mp3\")):\n            if i >= limit:\n                break\n            features.append(mfcc_extract(fn))\n            labels.append(label)\n            i+=1\n    return features, labels","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n\ntrain_cat_dirs = glob.glob(train_dir+'/*')\ntrain_cat = []\nfor cat_dir in train_cat_dirs:\n    tmp = cat_dir.split('/')[-1]\n    train_cat.append(tmp)\nprint('the number of kinds:', len(train_cat))\n\nclass_num = len(train_cat)\nfeatures, labels = parse_audio_files(train_dir, train_cat, LIMIT)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(len(features))\nprint(features[0].shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# plot few features\n\nfig = plt.figure(figsize=(28,24))\nfor i,mfcc in enumerate(tqdm(features[:100])):\n    if i%40 < 3 : \n        sub = plt.subplot(10,3,i%40+3*(i/40)+1)\n        librosa.display.specshow(mfcc,vmin=-700,vmax=300)\n        if ((i%40+3*(i/40)+1)%3==0) : \n            plt.colorbar()\n        sub.set_title(train_cat[labels[i]])\nplt.show()  ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_submission = pd.read_csv('../input/birdsong-recognition/sample_submission.csv')\ndf_submission.to_csv('submission.csv', index = None)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}