{"cells":[{"metadata":{},"cell_type":"markdown","source":"\nThanks to @rohanrao, because now we have additional datasets for training:\n\nhttps://www.kaggle.com/rohanrao/xeno-canto-bird-recordings-extended-a-m <br>\nhttps://www.kaggle.com/rohanrao/xeno-canto-bird-recordings-extended-n-z\n\nBut there are 14 columns missing, so we need to get somehow the data to fill them.<br> \n\nLet's check how we can do it. Hope it will be useful.","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-output":true,"_kg_hide-input":false},"cell_type":"code","source":"!pip install tinytag","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-output":true,"_kg_hide-input":false,"trusted":true},"cell_type":"code","source":"!pip install mutagen","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-output":true,"_kg_hide-input":false},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\n\nimport mutagen\nfrom mutagen.mp3 import MP3\nfrom tinytag import TinyTag\n\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"I found two nice libraries which can extract metadata from mp3:\n\n### MUTAGEN ###\n\nhttps://pypi.org/project/mutagen/\n\n### TINY TAG ###\n\nhttps://pypi.org/project/tinytag/\n\nAnd in this notebook I used both of them.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"DATA_PATH = '../input/birdsong-recognition/'\nAUDIO_PATH = \"../input/birdsong-recognition/train_audio\"\n\nDATA_PATH_EXT_A_M = '../input/xeno-canto-bird-recordings-extended-a-m/'\nAUDIO_PATH_EXT_A_M = \"../input/xeno-canto-bird-recordings-extended-a-m/A-M\"\n\nDATA_PATH_EXT_N_Z = '../input/xeno-canto-bird-recordings-extended-n-z/'\nAUDIO_PATH_EXT_N_Z = '../input/xeno-canto-bird-recordings-extended-n-z/N-Z'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's compare new datasets with old one","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train = pd.read_csv(DATA_PATH + 'train.csv')\ndf_train_ext_a_m = pd.read_csv(DATA_PATH_EXT_A_M + 'train_extended.csv')\ndf_train_ext_n_z = pd.read_csv(DATA_PATH_EXT_N_Z + 'train_extended.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Original dataset\nfor row in df_train.columns:\n    print(row)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#New dataset from A to M\nfor row in df_train_ext_a_m.columns:\n    print(row)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#New dataset from N to Z\nfor row in df_train_ext_n_z.columns:\n    print(row)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"a = df_train.columns\nb = df_train_ext_a_m.columns\nc = df_train_ext_n_z.columns","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"set(a).difference(b)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Just to make sure that there is no difference between A-M and N-Z parts\nset(b).difference(c)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So, as we can see 14 columns are missing.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Mutagen ###\nWith Mutagen we can get data for columns: bitrate_of_mp3, length, channels. Let's do it for one file.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"#Common info\nmutagen.File(\"../input/xeno-canto-bird-recordings-extended-a-m/A-M/aldfly/XC133197.mp3\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Only what we need\naudio = MP3(\"../input/xeno-canto-bird-recordings-extended-a-m/A-M/aldfly/XC133197.mp3\")\nprint(audio.info.bitrate)\nprint(audio.info.length)\nprint(audio.info.channels)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### TinyTag ###\n\nTinyTag alows to extract data for columns: background, description, primary_label, sampling_rate, secondary_labels, title. Most of the data stored at tag.comment, so it is necessary to work with strings a little bit. By the way there are more useful tags for extraction, which you can find in docs.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"tag = TinyTag.get(\"../input/xeno-canto-bird-recordings-extended-a-m/A-M/aldfly/XC133197.mp3\", image=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('file comment', tag.comment)\nprint('samples per second', tag.samplerate)\nprint('title of the song', tag.title)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Rest columns ###\n\nThe rest columns are: number_of_notes, pitch, rating, speed, volume. Let's check what values they have.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train['volume'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train['number_of_notes'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train['pitch'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train['rating'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train['speed'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"All of these remaining columns are secondary of importance. I believe that they can simply be filled with standard methods for filling data gaps. <br>\n\n<strong>So here are methods, which you can use for metadata extraction straight from the file to get more training data or to use in some other way.</strong>","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}