{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Extracted Dataset:\nThe Feature images extracted in below code can be found in [this Dataset](https://www.kaggle.com/timothyalexjohn/birdcall-spectrograms-cornell-birdcall-challenge)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Notebook_1 || [](http://)Part 1\n\nThis Notebook is the **1st Part** of 'Birdsong_Classifier_Keras_CNN' Notebook_1 for \"Cornell Birdcall Identification\" challenge.[](http://)\n\nIn this Part of Notebook_1 we convert Audio Files to Spectrogram Image Files using Librosa. \n\nIn the [Next Part](https://www.kaggle.com/timothyalexjohn/birdsong-classifier-keras-cnn-part-2-notebook-1) we upload this Notebook's Output and train CNNs on the extracted features. \n\n\n\n\n\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Notebooks in this Challenge:\n\n* Notebook_1:\n       \n > [Part 1](https://www.kaggle.com/timothyalexjohn/birdsong-classifier-keras-cnn-part-1-notebook-1):  Converting Audio Files to Spectrogram Image Files using Librosa\n  \n > [Part 2](https://www.kaggle.com/timothyalexjohn/birdsong-classifier-keras-cnn-part-2-notebook-1):  Running CNN on Extracted Features / Training\n \n > [Part 3](https://www.kaggle.com/timothyalexjohn/birdsong-classifier-keras-cnn-part-3-notebook-1): Prediction of test data\n \n\n* Notebook_2:\n        \n >Running LSTM Cells directly on Audio Files\n \n \n...Links will be updated soon!","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Importing Library","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import librosa         # Audio Manipulation Library\nimport librosa.display\nimport numpy as np\nimport pandas as pd\nimport os\nimport matplotlib.pyplot as plt","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Reference Directory","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"train_dir = '../input/birdsong-recognition/train_audio/'\ntrain_csv_dir = '../input/birdsong-recognition/train.csv'\n\ntest_dir = '../input/birdsong-recognition/test_audio/'\ntest_csv_dir = '../input/birdsong-recognition/test.csv'\n\ntrain_df = pd.read_csv(train_csv_dir)\ntest_df = pd.read_csv(test_csv_dir)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Saving Spectrogram to run CNN later..\n\nNote for test_audio, the recordings are from 3 sites. If the site is 'site_1' or 'site_2' we need recordings of 5 sec window. If from 'site_3' we use the whole recording as hinted in the data description page.\n\nRefer:\n[This Notebook](https://www.kaggle.com/cwthompson/birdsong-making-a-prediction) and\n[Discussion](https://www.kaggle.com/c/birdsong-recognition/discussion/158987)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_filesave_dir = '/kaggle/working/train/' \ntest_filesave_dir = '/kaggle/working/test/'\n\ndef create_spectrogram(df, start_time, duration, mode):\n    \n    if mode== 'train':\n        filepath = (train_dir +df[2] +'/' +df[7])\n\n    if mode== 'test':\n        filepath = (test_dir +df[3] +'.mp3') \n        \n    if mode== 'exa_test':                  \n        file = '_'.join(df[0].split('_')[:-1])\n        if file=='BLKFR-10-CPL_20190611_093000':\n            filepath = (exa_test_dir +file +'.pt540.mp3')\n        if file=='ORANGE-7-CAP_20190606_093000':\n            filepath = (exa_test_dir +file +'.pt623.mp3')\n    \n    try:\n        fig = plt.figure(figsize=[2.7,2.7])\n        filename = filepath.split('/')[-1].split('.')[0]\n        clip, sample_rate = librosa.load(filepath, sr=None, offset=start_time, duration=duration)\n        S = librosa.feature.melspectrogram(y=clip, sr=sample_rate)\n        librosa.display.specshow(librosa.power_to_db(S, ref=np.max))\n\n        if mode== 'train':\n            if not os.path.exists(train_filesave_dir +df[2]):\n                os.makedirs(train_filesave_dir +df[2] +'/')\n            plt.savefig((train_filesave_dir + df[2] +'/' +filename +'.jpg'), bbox_inches='tight',pad_inches=0, facecolor='black')\n        if mode== 'test':\n            if not os.path.exists(test_filesave_dir):\n                os.makedirs(test_filesave_dir) \n            plt.savefig((test_filesave_dir +filename +'.jpg'), bbox_inches='tight',pad_inches=0, facecolor='black')\n        if mode== 'exa_test':\n            if not os.path.exists(exa_test_filesave_dir):\n                os.makedirs(exa_test_filesave_dir)\n            plt.savefig((exa_test_filesave_dir + df[0] +'.jpg'), bbox_inches='tight',pad_inches=0, facecolor='black')\n            \n        fig.clear()          \n        plt.close(fig)\n        plt.close()\n        plt.close('all')     # These Lines are Very Important!! If not given, Server will run out of allocated Memory\n        plt.cla()\n        fig.clf()\n        plt.clf()\n        plt.close()\n    \n    except:\n        print(\"found a broken Audio File\")      ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Getting Rid of Un-Wanted Warnings when Loading Audio Files\nimport warnings\n\nwarnings.filterwarnings(\"ignore\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true},"cell_type":"code","source":"start_time= 0\n\nfor row in train_df.values:\n    create_spectrogram(row, start_time, row[6], mode= 'train')\n\nfor row in test_df.values:\n    if row[0]=='site_3' :\n        create_spectrogram(row, start_time, duration= None, mode= 'test')\n    else :\n        start_time = row[2] - 5\n        create_spectrogram(row, start_time, duration= 5, mode= 'test')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Trying on example_test_audio\n\nexa_test_dir = '../input/birdsong-recognition/example_test_audio/'\nexa_test_csv_dir = '../input/birdsong-recognition/example_test_audio_summary.csv'\nexa_test_df = pd.read_csv(exa_test_csv_dir)\n\nexa_test_filesave_dir = '/kaggle/working/exa_test/' \n \nfor row in exa_test_df.values:\n    if pd.isna(row[1])==False :\n        start_time = row[3] - 5\n        create_spectrogram(row, start_time, duration= 5, mode= 'exa_test')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### ZIP the Image Files","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import shutil\n\nshutil.make_archive('train_zipped', 'zip', '/kaggle/working/train')\nshutil.make_archive('test_zipped', 'zip', '/kaggle/working/test')\nshutil.make_archive('exa_test_zipped', 'zip', '/kaggle/working/exa_test')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Reference Notebooks\n* [Birdsong Making a Prediction](https://www.kaggle.com/cwthompson/birdsong-making-a-prediction)\n* [Simple Audio to Spectrum](https://www.kaggle.com/pawan28a95/simple-audio-to-spectrum-images)\n*  [Birdsong Keras Starter](https://www.kaggle.com/ryches/birdsong-keras-starter)\n\n\n# Note\n\nDownload the Ouput Image files and upload to the [Next Notebook](https://www.kaggle.com/timothyalexjohn/birdsong-classifier-keras-cnn-part-2-notebook-1) (Notebook_1 -Part 2) \n> It takes ~ 9 hrs to extract features from the Audio Folders. Suggested to download the Image Feature Folder from the Input of next Notebook\n\nSee You There!!    ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Please Upvote if you liked this Notebook!! ","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}