{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# **<span style='color:#A80808'>🎯 Goal</span>**\n\nIdentify bird species by sound: process continuous audio data and then acoustically recognize the species.\n\n# **<span style='color:#A80808'>🔑 Metric</span>**\nSubmissions are evaluated on a metric that is most similar to the macro F1 score. Given the amount of audio data used in this competition it wasn't feasible to label every single species found in every soundscape. Instead only a subset of species are actually scored for any given audio file. After dropping all of the un-scored rows we technically run a weighted classification accuracy with the weights set such that all of the species are assigned the same total weight and the true negatives and true positives for each species have the same weight. The extra complexity exists purely to allow us to have a great deal of control over which birds are scored for a given soundscape. For offline cross validation purposes, the macro F1 is the closest analogue to the actual metric.\n\n\n\n\n![](https://www.allaboutbirds.org/news/wp-content/uploads/2020/04/RBGull-Vyn-FI-1280x720.jpg)\n[Source](https://www.allaboutbirds.org/news/how-to-photograph-birds-in-flight/)","metadata":{}},{"cell_type":"markdown","source":"# **<span style='color:#A80808'>💾 Data</span>**\n\n## train_metadata.csv\n\nA wide range of metadata is provided for the training data. The most directly relevant fields are:\n- primary_label - a code for the bird species. You can review detailed information about the bird codes by appending the code to https://ebird.org/species/, such as https://ebird.org/species/amecro for the American Crow.\n- secondary_labels: Background species as annotated by the recordist. An empty list does not mean that no background birds are audible.\n- author - the eBird user who provided the recording.\n- filename: the associated audio file.\n- rating: Float value between 0.0 and 5.0 as an indicator of the quality rating on Xeno-canto and the number of background species, where 5.0 is the highest and 1.0 is the lowest. 0.0 means that this recording has no user rating yet.\n\n## train_audio/ \n- The bulk of the training data consists of short recordings of individual bird calls generously uploaded by users of xenocanto.org. These files have been downsampled to 32 kHz where applicable to match the test set audio and converted to the ogg format.\n\n## test_soundscapes/ \n- When you submit a notebook, the test_soundscapes directory will be populated with approximately 5,500 recordings to be used for scoring. These are each within a few milliseconds of 1 minute long and in the ogg audio format. Only one soundscape is available for download.\n\n## test.csv \n\nMetadata for the test set. Only the first three rows are available for download; the full test.csv is provided in the hidden test set.\n\n- row_id - A unique identifier for the row.\n- file_id - A unique identifier for the audio file.\n- bird - The ebird code for the row. There is one row for each of the scored species per 5 second window per audio file.\n- end_time - The last second of the 5 second time window (5, 10, 15, etc).\n\n## sample_submission.csv \nA valid sample submission. Only the first three rows are available for download; the full submission.csv is provided in the hidden test set.\n\n- row_id - A unique identifier for the row.\n- target - True/False for whether or not the bird in question called during the 5 second window.\n- scored_birds.json - The subset of the species in the dataset that are scored.\n\n## eBird_Taxonomy_v2021.csv \nData on the relationships between different species.","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport matplotlib.pyplot as plt\nimport plotly.express as px\nimport random\n\nfrom IPython.display import Image\nimport librosa\nimport librosa.display\nimport IPython.display as ipd","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-17T21:21:43.399538Z","iopub.execute_input":"2022-02-17T21:21:43.400019Z","iopub.status.idle":"2022-02-17T21:21:45.147802Z","shell.execute_reply.started":"2022-02-17T21:21:43.399891Z","shell.execute_reply":"2022-02-17T21:21:45.146767Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **<span style='color:#A80808'>Train metadata</span>**","metadata":{}},{"cell_type":"code","source":"train_metadata = pd.read_csv('../input/birdclef-2022/train_metadata.csv')\nprint(f'Shape of train_metadata: {train_metadata.shape}')\ntrain_metadata.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:21:45.149384Z","iopub.execute_input":"2022-02-17T21:21:45.149602Z","iopub.status.idle":"2022-02-17T21:21:45.288070Z","shell.execute_reply.started":"2022-02-17T21:21:45.149575Z","shell.execute_reply":"2022-02-17T21:21:45.287243Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Location","metadata":{}},{"cell_type":"code","source":"#https://www.kaggle.com/prokaggler/birdclef-2022-eda\nfig = px.scatter_geo(\n    train_metadata,\n    lat=\"latitude\",\n    lon=\"longitude\",\n    color=\"common_name\",\n    width=1000,\n    height=500,\n    title=\"BirdCLEF 2022 Training Data Location Plot\",\n)\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:23:09.916306Z","iopub.execute_input":"2022-02-17T21:23:09.916845Z","iopub.status.idle":"2022-02-17T21:23:10.534715Z","shell.execute_reply.started":"2022-02-17T21:23:09.916803Z","shell.execute_reply":"2022-02-17T21:23:10.533803Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Spacies","metadata":{}},{"cell_type":"code","source":"\nprint(f'There are {train_metadata.primary_label.nunique()} bird species:\\n{train_metadata.primary_label.unique()}')","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:21:45.289500Z","iopub.execute_input":"2022-02-17T21:21:45.289820Z","iopub.status.idle":"2022-02-17T21:21:45.312449Z","shell.execute_reply.started":"2022-02-17T21:21:45.289771Z","shell.execute_reply":"2022-02-17T21:21:45.311496Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pie, ax = plt.subplots(figsize=[20,12])\ntrain_metadata.groupby('primary_label').size().plot(kind='pie',\n                                                    #autopct='%.2f',\n                                                    ax=ax,\n                                                    title='Spacies distibution',\n                                                    rotatelabels =True,\n                                                    cmap = 'hot')\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:21:45.314990Z","iopub.execute_input":"2022-02-17T21:21:45.315342Z","iopub.status.idle":"2022-02-17T21:21:46.928330Z","shell.execute_reply.started":"2022-02-17T21:21:45.315295Z","shell.execute_reply":"2022-02-17T21:21:46.927425Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(20,10))\nplt.bar(train_metadata.primary_label.unique(), height=train_metadata.groupby('primary_label').size())\nplt.xticks(rotation=90)\nplt.title('Spacies distribution')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:21:46.929417Z","iopub.execute_input":"2022-02-17T21:21:46.929852Z","iopub.status.idle":"2022-02-17T21:21:48.916382Z","shell.execute_reply.started":"2022-02-17T21:21:46.929817Z","shell.execute_reply":"2022-02-17T21:21:48.915169Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(10,7))\ntrain_metadata.groupby('primary_label').size().hist(bins=100)\nplt.xlabel('Unique values')\nplt.ylabel('Frequence')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:21:48.917906Z","iopub.execute_input":"2022-02-17T21:21:48.918146Z","iopub.status.idle":"2022-02-17T21:21:49.301579Z","shell.execute_reply.started":"2022-02-17T21:21:48.918117Z","shell.execute_reply":"2022-02-17T21:21:49.300665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Background spacies","metadata":{}},{"cell_type":"code","source":"print(f'There are {train_metadata.secondary_labels.nunique()-1} background specie groups:\\n {train_metadata.secondary_labels.unique()}')","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:21:49.302979Z","iopub.execute_input":"2022-02-17T21:21:49.303230Z","iopub.status.idle":"2022-02-17T21:21:49.314570Z","shell.execute_reply.started":"2022-02-17T21:21:49.303198Z","shell.execute_reply":"2022-02-17T21:21:49.313611Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Authors","metadata":{}},{"cell_type":"code","source":"print(f'There are {train_metadata.author.nunique()} authors:\\n{train_metadata.author.unique()}')","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:21:49.316364Z","iopub.execute_input":"2022-02-17T21:21:49.316738Z","iopub.status.idle":"2022-02-17T21:21:49.325933Z","shell.execute_reply.started":"2022-02-17T21:21:49.316690Z","shell.execute_reply":"2022-02-17T21:21:49.325228Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(10,7))\ntrain_metadata.groupby('author').size().hist(bins=1000)\nplt.xlabel('Author unique values')\nplt.ylabel('Frequence')\nplt.yscale('log')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:21:49.327201Z","iopub.execute_input":"2022-02-17T21:21:49.327625Z","iopub.status.idle":"2022-02-17T21:21:51.973539Z","shell.execute_reply.started":"2022-02-17T21:21:49.327593Z","shell.execute_reply":"2022-02-17T21:21:51.972624Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"top_author = train_metadata.groupby('author').size().max()\nprint(f'Number of samples by the top author: {top_author}')","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:21:51.975957Z","iopub.execute_input":"2022-02-17T21:21:51.976321Z","iopub.status.idle":"2022-02-17T21:21:51.986349Z","shell.execute_reply.started":"2022-02-17T21:21:51.976283Z","shell.execute_reply":"2022-02-17T21:21:51.985374Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Audio files","metadata":{}},{"cell_type":"code","source":"print(f'There are {train_metadata.filename.nunique()} audio files:\\n{train_metadata.filename.unique()}')","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:21:51.987765Z","iopub.execute_input":"2022-02-17T21:21:51.987996Z","iopub.status.idle":"2022-02-17T21:21:52.004670Z","shell.execute_reply.started":"2022-02-17T21:21:51.987968Z","shell.execute_reply":"2022-02-17T21:21:52.003767Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://cdn.download.ams.birds.cornell.edu/api/v1/asset/97647371/1200)\n\n[afrsil1](https://ebird.org/species/afrsil1?siteLanguage=fr)","metadata":{}},{"cell_type":"code","source":"#https://www.kaggle.com/amandanavine/hawaiian-bird-species\nx, sr = librosa.load('../input/birdclef-2022/train_audio/afrsil1/XC125458.ogg', duration=20)\n%matplotlib inline\nX = librosa.stft(x)\nXdb = librosa.amplitude_to_db(abs(X))\nplt.figure(figsize=(14, 5))\nlibrosa.display.specshow(Xdb, cmap='hot', sr=sr, x_axis='s', y_axis='hz')\nipd.Audio('../input/birdclef-2022/train_audio/afrsil1/XC125458.ogg')","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:33:38.872631Z","iopub.execute_input":"2022-02-17T21:33:38.873057Z","iopub.status.idle":"2022-02-17T21:33:40.054634Z","shell.execute_reply.started":"2022-02-17T21:33:38.873014Z","shell.execute_reply":"2022-02-17T21:33:40.053920Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://abcbirds.org/wp-content/uploads/2017/02/Akekee-_Jim-Denny.jpg)\n\n[akekee](https://abcbirds.org/bird/akekee/)","metadata":{}},{"cell_type":"code","source":"x, sr = librosa.load('../input/birdclef-2022/train_audio/akekee/XC174953.ogg', duration=20)\n%matplotlib inline\nX = librosa.stft(x)\nXdb = librosa.amplitude_to_db(abs(X))\nplt.figure(figsize=(14, 5))\nlibrosa.display.specshow(Xdb, cmap='hot', sr=sr, x_axis='s', y_axis='hz')\nipd.Audio('../input/birdclef-2022/train_audio/akekee/XC174953.ogg')","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:33:50.640181Z","iopub.execute_input":"2022-02-17T21:33:50.641057Z","iopub.status.idle":"2022-02-17T21:33:51.024351Z","shell.execute_reply.started":"2022-02-17T21:33:50.641015Z","shell.execute_reply":"2022-02-17T21:33:51.023241Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://cdn.download.ams.birds.cornell.edu/api/v1/asset/96737001/900)\n\n[akepa1](https://ebird.org/species/akepa1/US)","metadata":{}},{"cell_type":"code","source":"x, sr = librosa.load('../input/birdclef-2022/train_audio/akepa1/XC122473.ogg', duration=20)\n%matplotlib inline\nX = librosa.stft(x)\nXdb = librosa.amplitude_to_db(abs(X))\nplt.figure(figsize=(14, 5))\nlibrosa.display.specshow(Xdb, cmap='hot', sr=sr, x_axis='s', y_axis='hz')\nipd.Audio('../input/birdclef-2022/train_audio/akepa1/XC122473.ogg')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Rating","metadata":{}},{"cell_type":"code","source":"print(f'There are {train_metadata.rating.nunique()} rating scores from 0 (no voting yet) to 5 (highest score):\\n{train_metadata.rating.unique()}')","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:21:52.006301Z","iopub.execute_input":"2022-02-17T21:21:52.007234Z","iopub.status.idle":"2022-02-17T21:21:52.014911Z","shell.execute_reply.started":"2022-02-17T21:21:52.007185Z","shell.execute_reply":"2022-02-17T21:21:52.014083Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pie, ax = plt.subplots(figsize=[20,12])\ntrain_metadata.groupby('rating').size().plot(kind='pie',\n                                                    #autopct='%.2f',\n                                                    ax=ax,\n                                                    title='Rating distibution',\n                                                    rotatelabels =True,\n                                                    cmap = 'hot')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:21:52.016400Z","iopub.execute_input":"2022-02-17T21:21:52.016900Z","iopub.status.idle":"2022-02-17T21:21:52.252796Z","shell.execute_reply.started":"2022-02-17T21:21:52.016867Z","shell.execute_reply":"2022-02-17T21:21:52.252152Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **<span style='color:#A80808'>🏆 Submission</span>**","metadata":{}},{"cell_type":"code","source":"submission = pd.read_csv('../input/birdclef-2022/sample_submission.csv')\nsubmission['target'] = bool(random.getrandbits(1))\nsubmission.to_csv('submission.csv', index=False)\nsubmission.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-17T21:21:52.253835Z","iopub.execute_input":"2022-02-17T21:21:52.254428Z","iopub.status.idle":"2022-02-17T21:21:52.276470Z","shell.execute_reply.started":"2022-02-17T21:21:52.254391Z","shell.execute_reply":"2022-02-17T21:21:52.275415Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# This notebook is under construction 🏗","metadata":{}}]}