{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Intro\nWelcome to the [Rainforest Connection Species Audio Detection](https://www.kaggle.com/c/rfcx-species-audio-detection/data) competition. \n![](https://storage.googleapis.com/kaggle-competitions/kaggle/21669/logos/header.png)\nWe will give you first a short introduction to start with your work. The nex step is to show a short analysis befor definen a model with keras.\n\nThese are the features of the train data:\n* recording_id: unique identifier for recording\n* species_id: unique identifier for species\n* songtype_id: unique identifier for songtype\n* t_min: start second of annotated signal\n* f_min: lower frequency of annotated signal\n* t_max: end second of annotated signal\n* f_max: upper frequency of annotated signal\n* is_tp: [tfrecords only] an indicator of whether the label is from the train_tp (1) or train_fp (0) file.\n\nWe recommend [this notebook](https://www.kaggle.com/drcapa/esc-50-eda-pytorch) for handling audio data.\n\n<span style=\"color: royalblue;\">Please vote the notebook up if it helps you. Feel free to leave a comment above the notebook. Thank you. </span>"},{"metadata":{},"cell_type":"markdown","source":"# Libraries\nWe load some standard libraries and packages of the keras library."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nimport matplotlib.pyplot as plt\nimport soundfile as sf\nimport librosa\nimport librosa.display\nimport IPython.display as display\n\nfrom keras.utils import Sequence\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Dropout, Flatten, Conv1D, MaxPool1D, BatchNormalization\nfrom keras.optimizers import RMSprop,Adam\nfrom keras.applications import VGG19, VGG16, ResNet50\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Path"},{"metadata":{"trusted":true},"cell_type":"code","source":"path = '/kaggle/input/rfcx-species-audio-detection/'\nos.listdir(path)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Functions\nWe define some helper functions for loading and visualization the data."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def read_flac_file(path, file):\n    \"\"\" Read flac audio file and return numpay array and samplerate\"\"\"\n    \n    data, samplerate = sf.read(path+file)\n    return data, samplerate\n\ndef plot_audio_file(data, samplerate, t_min, t_max, species):\n    \"\"\" Plot the cutout for the speciec label \"\"\"\n    \n    sr = samplerate\n    fig = plt.figure(figsize=(10, 6))\n    x = range(len(data))\n    y = data\n    plt.plot(x, y)\n    x = range(int(t_min*sr), int(t_max*sr))\n    y = data[int(t_min*sr):int(t_max*sr)]\n    plt.plot(x, y, color='red', label = 'species '+str(species))\n    plt.legend(loc='upper center')\n    plt.grid()\n    \ndef plot_spectrogram(data, samplerate, t_min, t_max):\n    \"\"\" Plot spectrogram with mel scaling \"\"\"\n    \n    sr = samplerate\n    data_sub = data[int(t_min*sr):int(t_max*sr)]\n    spectrogram = librosa.feature.melspectrogram(data_sub, sr=sr)\n    log_spectrogram = librosa.power_to_db(spectrogram, ref=np.max)\n    librosa.display.specshow(log_spectrogram, sr=sr, x_axis='time', y_axis='mel')\n    \ndef plot_bar_compare(data1, data2, name, rot=False):\n    \"\"\" Compare the distribution between train_fp and train_tp data \"\"\"\n    \n    fig, axs = plt.subplots(1, 2, figsize=(9, 3), sharey=True)\n    \n    data1_label = data1[name].value_counts().sort_index()\n    dict_data1 = dict(zip(data1_label.keys(), ((100*(data1_label)/len(data1.index)).tolist())))\n    data1_names = list(dict_data1.keys())\n    data1_values = list(dict_data1.values())\n    \n    data2_label = data2[name].value_counts().sort_index()\n    dict_data2 = dict(zip(data2_label.keys(), ((100*(data2_label)/len(data2.index)).tolist())))\n    data2_names = list(dict_data2.keys())\n    data2_values = list(dict_data2.values())\n    \n    axs[0].bar(data1_names, data1_values, color='yellowgreen')\n    axs[1].bar(data2_names, data2_values, color='sandybrown')\n    axs[0].grid()\n    axs[1].grid()\n    axs[0].set_title('train_fp')\n    axs[1].set_title('train_tp')\n    axs[0].set_ylabel('%')\n    if(rot==True):\n        axs[0].set_xticklabels(data1_names, rotation=45)\n        axs[1].set_xticklabels(data2_names, rotation=45)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","collapsed":true,"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":false},"cell_type":"markdown","source":"# Load Data"},{"metadata":{},"cell_type":"markdown","source":"csv File:"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_fp = pd.read_csv(path+'train_fp.csv')\ntrain_tp = pd.read_csv(path+'train_tp.csv')\nsamp_subm = pd.read_csv(path+'sample_submission.csv')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"audio File:"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_audio_files = os.listdir(path+'train')\ntest_audio_files = os.listdir(path+'test')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Load example audio file:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"data, samplerate = read_flac_file(path+'train/', train_audio_files[0])\nprint('data array:', data)\nprint('samplerate:', samplerate) \nprint('number of data values:', len(data))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# EDA"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"print('number of false positive:', len(train_fp))\nprint('number of true positive:', len(train_tp))\nprint('number of samp_subm rows:', len(samp_subm))\nprint('number of train audio files:', len(train_audio_files))\nprint('number of test audio files:', len(test_audio_files))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Distribution of the feature species_id:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"plot_bar_compare(train_fp, train_tp, 'species_id', rot=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Distribution of the feature songtype_id:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"plot_bar_compare(train_fp, train_tp, 'songtype_id', rot=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"There could be more than one species for one audio file:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_fp[0:3]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Discribtsion of the features:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_fp.describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# A Sample File"},{"metadata":{"trusted":true},"cell_type":"code","source":"recording_id = '00204008d'\ndata, samplerate = read_flac_file(path+'train/', recording_id+'.flac')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"display.Audio(path+'train/'+recording_id+'.flac')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"t_min = train_fp[train_fp['recording_id']==recording_id]['t_min'][0]\nt_max = train_fp[train_fp['recording_id']==recording_id]['t_max'][0]\nlabel = train_fp[train_fp['recording_id']==recording_id]['species_id'][0]\nplot_audio_file(data, samplerate, t_min, t_max, label)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Plot [spectrogram](https://en.wikipedia.org/wiki/Spectrogram) with mel scaling:"},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_spectrogram(data, samplerate, t_min, t_max)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Prepare Data For Model\n## Train Lables"},{"metadata":{"trusted":true},"cell_type":"code","source":"y_train_index = [file.split('.')[0] for file in train_audio_files]\ny_train_columns = ['s'+str(i) for i in range(24)]\ny_train = pd.DataFrame(0, index=y_train_index, columns=y_train_columns)\n\nfor row in train_fp.index:\n    index = train_fp.loc[row, 'recording_id']\n    column = 's'+str(train_fp.loc[row, 'species_id'])\n    y_train.loc[index, column] = 1\n\nfor row in train_tp.index:\n    index = train_tp.loc[row, 'recording_id']\n    column = 's'+str(train_tp.loc[row, 'species_id'])\n    y_train.loc[index, column] = 1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"y_train.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Audio Data Generator"},{"metadata":{"trusted":true},"cell_type":"code","source":"class DataGenerator(Sequence):\n    def __init__(self, path, list_IDs, labels, batch_size):\n        self.path = path\n        self.list_IDs = list_IDs\n        self.labels = labels\n        self.batch_size = batch_size\n        self.indexes = np.arange(len(self.list_IDs))\n        \n    def __len__(self):\n        len_ = int(len(self.list_IDs)/self.batch_size)\n        if len_*self.batch_size < len(self.list_IDs):\n            len_ += 1\n        return len_\n    \n    def __getitem__(self, index):\n        indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size]\n        list_IDs_temp = [self.list_IDs[k] for k in indexes]\n        X, y = self.__data_generation(list_IDs_temp)\n        X = X.reshape((self.batch_size, 1000, 2880//2))\n        return X, y\n    \n    def __data_generation(self, list_IDs_temp):\n        X = np.zeros((self.batch_size, 2880000//2))\n        y = np.zeros((self.batch_size, 24))\n        for i, ID in enumerate(list_IDs_temp):\n            audio_file, audio_sr = read_flac_file(self.path, ID)\n            audio_file_fft = data_fft = np.abs(np.fft.fft(audio_file)[: len(audio_file)//2])\n            # scale data\n            audio_file_fft = (audio_file_fft-audio_file_fft.mean())/audio_file_fft.std()\n            X[i, ] = audio_file_fft\n            y[i, ] = self.labels.loc[ID.split('.')[0]]\n        return X, y","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Define Model"},{"metadata":{"trusted":true},"cell_type":"code","source":"epochs = 15\nlernrate = 2e-3","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model = Sequential()\nmodel.add(Conv1D(128, input_shape=(1000, 2880//2,), kernel_size=5, strides=4, activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool1D(pool_size=(4)))\nmodel.add(Conv1D(128, kernel_size=3, activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool1D(pool_size=(4)))\nmodel.add(Conv1D(128, kernel_size=3, activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool1D(pool_size=(4)))\nmodel.add(Conv1D(256, kernel_size=3, activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(Flatten())\nmodel.add(Dense(512, activation='relu'))\nmodel.add(Dense(24, activation='softmax'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.compile(optimizer = Adam(lr=lernrate),\n              loss='binary_crossentropy',\n              metrics=['binary_accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Train Model"},{"metadata":{"trusted":true},"cell_type":"code","source":"batch_size = 64","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_generator = DataGenerator(path+'train/', train_audio_files, y_train, batch_size)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"history = model.fit_generator(generator=train_generator,\n                              epochs = epochs,\n                              workers=4)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Predict Test Data"},{"metadata":{"trusted":true},"cell_type":"code","source":"y_test = pd.read_csv(path+'sample_submission.csv', index_col=0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_generator = DataGenerator(path+'test/', test_audio_files, y_test, batch_size)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"y_pred = model.predict_generator(test_generator, verbose=1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Write Output"},{"metadata":{"trusted":true},"cell_type":"code","source":"output = pd.DataFrame(y_pred, columns = samp_subm.columns[1:25])\noutput.insert(0, 'recording_id', samp_subm['recording_id'])\noutput.dropna(inplace=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Export Data"},{"metadata":{"trusted":true},"cell_type":"code","source":"output.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}