![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1659719%2F0828f590b68790f3cdaeda61b07d8da9%2Fmy-awesome-meme.jpeg?generation=1598644353014676&alt=media\")
I have noticed some of ebird_code presented inside example_test_audio_summary.csv file but missed inside train_audio folder. There are 5 such cases:
\n66(!) rows from 153 consist one of these unknown labels and I've saved such cases here. Do we know for sure that it doesn't happen for hidden test set?
", "messages": [ { "id": "989457", "postDate": "08/28/2020 19:54:44", "content": "
I have noticed some of ebird_code presented inside example_test_audio_summary.csv file but missed inside train_audio folder. There are 5 such cases:
\n66(!) rows from 153 consist one of these unknown labels and I've saved such cases here. Do we know for sure that it doesn't happen for hidden test set?
", "rawMarkdown": "\n\nI have noticed some of *ebird_code* presented inside **example_test_audio_summary.csv** file but missed inside **train_audio** folder. There are 5 such cases:\n- squirrel\n- mouqua\n- whhwoo\n- hawo\n- unk\n\n66(!) rows from 153 consist one of these unknown labels and I've saved such cases [here](https://www.kaggle.com/koza4ukdmitrij/lost-birds-from-cornell-birdcall-identification). Do we know for sure that it doesn't happen for hidden test set?", "votes": null }, { "id": "989498", "postDate": "08/28/2020 20:59:02", "content": "Yes, the example annotations are from another project, and have some extra labels that don't appear in the test set. We decided to leave them in as a reminder that the world contains non-bird entities. :)
", "rawMarkdown": "Yes, the example annotations are from another project, and have some extra labels that don't appear in the test set. We decided to leave them in as a reminder that the world contains non-bird entities. :)", "votes": null }, { "id": "990178", "postDate": "08/29/2020 12:07:19", "content": "Thank you for response! But can we use this audio for better understanding bird's voices (that presence inside training set), enviroment sounds and so on, or these sounds were taken from absolutely different distribution?
", "rawMarkdown": "Thank you for response! But can we use this audio for better understanding bird's voices (that presence inside training set), enviroment sounds and so on, or these sounds were taken from absolutely different distribution?", "votes": null }, { "id": "990187", "postDate": "08/29/2020 12:19:22", "content": "Hello @koza4ukdmitrij
\nI found exactly same and about to post a discussion.
\n@tomdenton this means that we cannot (and should not) use example_test_audio to
\nevaluate our models. I appreciate if you could confirm if I understand correct.
one more question: 'site1' and 'site2' of example_test_audio are not identical
\nto 'site1' and 'site2' of actual test_audio recording?
Thank you,
\nmeg.
Thank you for the information! But I think it would be beneficial to check your model at least on existing inside train set birds. Anyway lets waiting for host response.
", "rawMarkdown": "Thank you for the information! But I think it would be beneficial to check your model at least on existing inside train set birds. Anyway lets waiting for host response.", "votes": null }, { "id": "990546", "postDate": "08/29/2020 17:25:55", "content": "\"[C]an we use this audio for better understanding bird's voices[?]\"
\nIn the sense that these clips are typical of an additional random forest (pun intended), yes! The heart of the problem we're working on is how to transfer learning from a large 'pretty good' training set to usage in the actual field; dealing gracefully with the domain shift is at the heart of what we're trying to do.
\n\"[Are] these sounds were taken from absolutely different distribution?\"
\nIt's not an 'absolutely different' distribution, but still North American soundscape audio containing bird vocalizations from a known set of species. What we're seeing in this problem space (here and in the BirdClef challenges) is 'lab-trained' models which are having a very difficult time generalizing to the real world, and we would love to know why. And if not 'why' at least how to do better…
", "rawMarkdown": "\"[C]an we use this audio for better understanding bird's voices[?]\"\n\nIn the sense that these clips are typical of an additional random forest (pun intended), yes! The heart of the problem we're working on is how to transfer learning from a large 'pretty good' training set to usage in the actual field; dealing gracefully with the domain shift is at the heart of what we're trying to do. \n\n\"[Are] these sounds were taken from absolutely different distribution?\"\n\nIt's not an 'absolutely different' distribution, but still North American soundscape audio containing bird vocalizations from a known set of species. What we're seeing in this problem space (here and in the BirdClef challenges) is 'lab-trained' models which are having a very difficult time generalizing to the real world, and we would love to know why. And if not 'why' at least how to do better...", "votes": null }, { "id": "990550", "postDate": "08/29/2020 17:28:20", "content": "As mentioned in the 'Data' description, the example audio is from a different site than the actual test audio.
", "rawMarkdown": "As mentioned in the 'Data' description, the example audio is from a different site than the actual test audio.", "votes": null }, { "id": "990560", "postDate": "08/29/2020 17:37:00", "content": "Thank you, really helpful!
", "rawMarkdown": "Thank you, really helpful!", "votes": null }, { "id": "990852", "postDate": "08/29/2020 22:41:23", "content": "Thank you so much @tomdenton for your clarification.
\n… just to double check, if there is any information that I miss (there seems lots of them).
\n\n\nAs mentioned in the 'Data' description, the example audio is from a different site than the actual test audio.
\n
Do you mean this passage below in Data description that tells the example test audio are from different sites
\nfrom actual test audio? Or is there any description that I miss?
\"Two example soundscapes from another data source are also provided to illustrate how the soundscapes are labeled and the hidden dataset folder structure. The two example audio files … \"
\n… just one more confirmation needed.
\n\n\nIt's not an 'absolutely different' distribution, but still North American soundscape audio containing bird vocalizations from a known set of species.
\n
Do I understand correct that the example test audio set can be used as bird call samples. I mean, they are labeled
\ncorrectly (correct enough, or at least with similar precisions with the training data that are given to us).
\nIn other words, the example test audio set are not just a place holder to show the layout of the test data set, but
\nexample test data do have meaningful information of birdcalls and their correct labels. Do I understand correct?
thank you for your patience,
\nmeg.
Yes, the example annotations are from another project, and have some extra labels that don't appear in the test set. We decided to leave them in as a reminder that the world contains non-bird entities. :)
", "votes": null, "replies": [ { "id": 990178, "author_name": "koza4ukdmitrij", "author_url": "", "post_date": "08/29/2020 12:07:19", "content": "Thank you for response! But can we use this audio for better understanding bird's voices (that presence inside training set), enviroment sounds and so on, or these sounds were taken from absolutely different distribution?
", "votes": null, "replies": [] }, { "id": 990546, "author_name": "tomdenton", "author_url": "", "post_date": "08/29/2020 17:25:55", "content": "\"[C]an we use this audio for better understanding bird's voices[?]\"
\nIn the sense that these clips are typical of an additional random forest (pun intended), yes! The heart of the problem we're working on is how to transfer learning from a large 'pretty good' training set to usage in the actual field; dealing gracefully with the domain shift is at the heart of what we're trying to do.
\n\"[Are] these sounds were taken from absolutely different distribution?\"
\nIt's not an 'absolutely different' distribution, but still North American soundscape audio containing bird vocalizations from a known set of species. What we're seeing in this problem space (here and in the BirdClef challenges) is 'lab-trained' models which are having a very difficult time generalizing to the real world, and we would love to know why. And if not 'why' at least how to do better…
", "votes": null, "replies": [] }, { "id": 990560, "author_name": "koza4ukdmitrij", "author_url": "", "post_date": "08/29/2020 17:37:00", "content": "Thank you, really helpful!
", "votes": null, "replies": [] } ] }, { "id": 990187, "author_name": "megner", "author_url": "", "post_date": "08/29/2020 12:19:22", "content": "Hello @koza4ukdmitrij
\nI found exactly same and about to post a discussion.
\n@tomdenton this means that we cannot (and should not) use example_test_audio to
\nevaluate our models. I appreciate if you could confirm if I understand correct.
one more question: 'site1' and 'site2' of example_test_audio are not identical
\nto 'site1' and 'site2' of actual test_audio recording?
Thank you,
\nmeg.
Thank you for the information! But I think it would be beneficial to check your model at least on existing inside train set birds. Anyway lets waiting for host response.
", "votes": null, "replies": [] }, { "id": 990550, "author_name": "tomdenton", "author_url": "", "post_date": "08/29/2020 17:28:20", "content": "As mentioned in the 'Data' description, the example audio is from a different site than the actual test audio.
", "votes": null, "replies": [] } ] }, { "id": 990852, "author_name": "megner", "author_url": "", "post_date": "08/29/2020 22:41:23", "content": "Thank you so much @tomdenton for your clarification.
\n… just to double check, if there is any information that I miss (there seems lots of them).
\n\n\nAs mentioned in the 'Data' description, the example audio is from a different site than the actual test audio.
\n
Do you mean this passage below in Data description that tells the example test audio are from different sites
\nfrom actual test audio? Or is there any description that I miss?
\"Two example soundscapes from another data source are also provided to illustrate how the soundscapes are labeled and the hidden dataset folder structure. The two example audio files … \"
\n… just one more confirmation needed.
\n\n\nIt's not an 'absolutely different' distribution, but still North American soundscape audio containing bird vocalizations from a known set of species.
\n
Do I understand correct that the example test audio set can be used as bird call samples. I mean, they are labeled
\ncorrectly (correct enough, or at least with similar precisions with the training data that are given to us).
\nIn other words, the example test audio set are not just a place holder to show the layout of the test data set, but
\nexample test data do have meaningful information of birdcalls and their correct labels. Do I understand correct?
thank you for your patience,
\nmeg.