Maybe I am just missing something but I can't find the taxonomy for the species only numeric ids.
\nCollecting external data (e.g. Xeno Canto) would be easier if we knew the species in advance.
Maybe I am just missing something but I can't find the taxonomy for the species only numeric ids.
\nCollecting external data (e.g. Xeno Canto) would be easier if we knew the species in advance.
Here is the list of the species: https://www.sciencedirect.com/science/article/pii/S1574954120300637
", "rawMarkdown": "Here is the list of the species: https://www.sciencedirect.com/science/article/pii/S1574954120300637", "votes": 2, "replies": [ { "id": 1200320, "postDate": "2021-02-14T15:23:39.673Z", "content": "hmm
\nbased on your link, i searched for papers related to this. to my surprise:
QUOTE
\nIn classifying a test set of manually validated positive and negative template-based detections, our proposed model achieves 97.7% sensitivity (true positive rate), 96.4% specificity (true negative rate) and 99.5% Area Under a Curve (AUC).
@hengck23 here is the repo for the above paper: https://github.com/microsoft/Multi_Species_Bioacoustic_Classification
", "rawMarkdown": "@hengck23 here is the repo for the above paper: https://github.com/microsoft/Multi_Species_Bioacoustic_Classification" } ] }, { "id": 1119904, "postDate": "2020-12-20T12:42:58.700Z", "content": "@jacklebien: Could you reveal the names of the species once the competition is over, please? I'd like to see the birds / frogs that make these sounds.
", "rawMarkdown": "@jacklebien: Could you reveal the names of the species once the competition is over, please? I'd like to see the birds / frogs that make these sounds.", "votes": 1, "replies": [ { "id": 1124101, "postDate": "2020-12-23T17:14:58.417Z", "content": "@jottbe Sure we can do this! I will try to remember but remind me if needed
", "rawMarkdown": "@jottbe Sure we can do this! I will try to remember but remind me if needed", "votes": 1, "isDeleted": true } ] }, { "id": 1082445, "postDate": "2020-11-17T22:33:44.797Z", "content": "Hello! We chose to hide the species names to discourage collection of more training data from external sources and encourage solutions for limited training data.
\nSolutions requiring minimal training data will accelerate the development of eco-acoustic monitoring systems, and especially help with rare species where training data is difficult to find.
", "rawMarkdown": "Hello! We chose to hide the species names to discourage collection of more training data from external sources and encourage solutions for limited training data. \n\nSolutions requiring minimal training data will accelerate the development of eco-acoustic monitoring systems, and especially help with rare species where training data is difficult to find.", "votes": 16, "isDeleted": true, "replies": [ { "id": 1082448, "postDate": "2020-11-17T22:41:40.347Z", "content": "Hi, will there be an explanation about songtype_id ?
I understand the reason for hiding the species names, it makes it just seem like a plainly focused ML task without learning a different topic e.g. the species names and how they sound, unfortunately.
", "rawMarkdown": "Hi, will there be an explanation about `songtype_id` ?\n\nI understand the reason for hiding the species names, it makes it just seem like a plainly focused ML task without learning a different topic e.g. the species names and how they sound, unfortunately.", "votes": 1 }, { "id": 1082463, "postDate": "2020-11-17T23:17:52.433Z", "content": "The evaluation is based only on predicting species presence (not songtype), but as indicated by songtype_id some species have two different songs in the training data. In these cases both songtypes are common signals from the species. Songtype 1 indicates the most common song as determined by a biologist on our team, songtype 4 just indicates another common song variation.
I understand there is less interesting biology to learn without species names, but currently we think preventing extra data collection is priority, and are interested in solutions that can generalize well to species outside our dataset
", "rawMarkdown": "The evaluation is based only on predicting species presence (not songtype), but as indicated by `songtype_id` some species have two different songs in the training data. In these cases both songtypes are common signals from the species. Songtype 1 indicates the most common song as determined by a biologist on our team, songtype 4 just indicates another common song variation.\n\nI understand there is less interesting biology to learn without species names, but currently we think preventing extra data collection is priority, and are interested in solutions that can generalize well to species outside our dataset\n", "votes": 9, "isDeleted": true }, { "id": 1088044, "postDate": "2020-11-23T09:11:18.707Z", "content": "@jacklebien Thank you for your explanation! I understood the reason why the species names are hidden. However, it is still allowed to use external audio data to train our models, right? I assume some participants (e.g. specialists in biology) might be able to find out the name of species. Then, they could collect the specific additional audio data easily, which the competition host does not want. Also, it might be unfair for the other participants.
\nSo, my question is, do you have any rule (or requirements) if participants find out the name of species? Thank you!
", "rawMarkdown": "@jacklebien Thank you for your explanation! I understood the reason why the species names are hidden. However, it is still allowed to use external audio data to train our models, right? I assume some participants (e.g. specialists in biology) might be able to find out the name of species. Then, they could collect the specific additional audio data easily, which the competition host does not want. Also, it might be unfair for the other participants.\n\nSo, my question is, do you have any rule (or requirements) if participants find out the name of species? Thank you!", "votes": 2 }, { "id": 1088047, "postDate": "2020-11-23T09:15:41.413Z", "content": "From the rules:
\n\n", "rawMarkdown": "From the rules:\n\n> External Data:\nYou may use data other than the Competition Data (“External Data”) to develop and test your Submissions. However, you will ensure the External Data is publicly available to use by all participants of the Competition for purposes of the competition at no cost to the other participants. The ability to use External Data under this Section 7.C (External Data) does not limit you other obligations under these Competition Rules, including but not limited to Section 11 (Winners Obligations). " }, { "id": 1088543, "postDate": "2020-11-23T17:59:51.490Z", "content": "External Data:
\n
\n You may use data other than the Competition Data (“External Data”) to develop and test your Submissions. However, you will ensure the External Data is publicly available to use by all participants of the Competition for purposes of the competition at no cost to the other participants. The ability to use External Data under this Section 7.C (External Data) does not limit you other obligations under these Competition Rules, including but not limited to Section 11 (Winners Obligations).
@hdsk38 Yes, it is permitted to use External data, as allowed by the rules. So if you were to decipher the species and/or find additional external data, it would be permitted. Hand-labeling, however, is not permitted, including if you incorporate hand-labeling within your model's training.
", "rawMarkdown": "@hdsk38 Yes, it is permitted to use External data, as allowed by the rules. So if you were to decipher the species and/or find additional external data, it would be permitted. Hand-labeling, however, is not permitted, including if you incorporate hand-labeling within your model's training." }, { "id": 1088571, "postDate": "2020-11-23T18:42:33.043Z", "content": "Can you clarify if hand-labeling and/or editing of training data is permitted?
\nIt is unclear for me if your post refers to total ban on hand-labeling or it just refers to hypothetical situation where someone hand-labels test data and adds it to model training data. In previous posts only hand-labeling test data was explicitly banned.
", "rawMarkdown": "Can you clarify if hand-labeling and/or editing of training data is permitted?\n\nIt is unclear for me if your post refers to total ban on hand-labeling or it just refers to hypothetical situation where someone hand-labels test data and adds it to model training data. In previous posts only hand-labeling test data was explicitly banned." }, { "id": 1143265, "postDate": "2021-01-07T20:18:31.447Z", "content": "@jacklebien you wrote:
\n\n\nWe chose to hide the species names to discourage collection of more training data from external sources and encourage solutions for limited training data.
\n
and
\n\n\nwe think preventing extra data collection is priority
\n
Then why is external data authorized as @juliaelliott reminds us?
", "rawMarkdown": "@jacklebien you wrote:\n> We chose to hide the species names to discourage collection of more training data from external sources and encourage solutions for limited training data. \n\nand \n\n> we think preventing extra data collection is priority\n\nThen why is external data authorized as @juliaelliott reminds us?", "votes": 1 }, { "id": 1143285, "postDate": "2021-01-07T20:32:16.617Z", "content": "I think the rules do not force the intent of the hosts…
\nFrom the rules:
\n\n\n\n
\n- COMPETITION ENTRY…
\n
\n Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.
I will be nice and Kaggley (as @inversion asked in the other thread) and I won't try to decipher the data and collect more and I obviously won't try to hand label the test dataset. However I might try to fix the training labels manually…
", "rawMarkdown": "I think the rules do not force the intent of the hosts...\n\nFrom the rules: \n\n> 5. COMPETITION ENTRY...\nSubmissions may not use or incorporate information from hand labeling or human prediction of the **validation dataset or test data records**. \n\nI will be nice and Kaggley (as @inversion asked in the other thread) and I won't try to decipher the data and collect more and I obviously won't try to hand label the test dataset. However I might try to fix the training labels manually...\n", "votes": 2 }, { "id": 1143421, "postDate": "2021-01-07T22:36:09.757Z", "content": "Hi, all. Sorry that these responses have caused some confusion. I have fully clarified with the host, and we are in agreement that: External data is allowed and taking actions to include external data — including that which makes use of hand-labeled species on the training data — is fully permitted, per the competition rules.
", "rawMarkdown": "Hi, all. Sorry that these responses have caused some confusion. I have fully clarified with the host, and we are in agreement that: **External data is allowed** and taking actions to include external data — including that which makes use of hand-labeled species on the training data — is fully permitted, per the competition rules.", "votes": 4 } ] } ], "comments": [ { "id": 1200197, "author_name": "tugstugi", "author_url": "", "post_date": "2021-02-14T13:40:39.823000", "content": "Here is the list of the species: https://www.sciencedirect.com/science/article/pii/S1574954120300637
", "votes": 2, "replies": [ { "id": 1200320, "author_name": "hengck23", "author_url": "", "post_date": "2021-02-14T15:23:39.673000", "content": "hmm
\nbased on your link, i searched for papers related to this. to my surprise:
QUOTE
\nIn classifying a test set of manually validated positive and negative template-based detections, our proposed model achieves 97.7% sensitivity (true positive rate), 96.4% specificity (true negative rate) and 99.5% Area Under a Curve (AUC).
@hengck23 here is the repo for the above paper: https://github.com/microsoft/Multi_Species_Bioacoustic_Classification
", "votes": 0, "replies": [] } ] }, { "id": 1119904, "author_name": "Jott Be", "author_url": "", "post_date": "2020-12-20T12:42:58.700000", "content": "@jacklebien: Could you reveal the names of the species once the competition is over, please? I'd like to see the birds / frogs that make these sounds.
", "votes": 1, "replies": [ { "id": 1124101, "author_name": "", "author_url": "", "post_date": "2020-12-23T17:14:58.417000", "content": "@jottbe Sure we can do this! I will try to remember but remind me if needed
", "votes": 1, "replies": [] } ] }, { "id": 1082445, "author_name": "", "author_url": "", "post_date": "2020-11-17T22:33:44.797000", "content": "Hello! We chose to hide the species names to discourage collection of more training data from external sources and encourage solutions for limited training data.
\nSolutions requiring minimal training data will accelerate the development of eco-acoustic monitoring systems, and especially help with rare species where training data is difficult to find.
", "votes": 16, "replies": [ { "id": 1082448, "author_name": "Ali Abdin", "author_url": "", "post_date": "2020-11-17T22:41:40.347000", "content": "Hi, will there be an explanation about songtype_id ?
I understand the reason for hiding the species names, it makes it just seem like a plainly focused ML task without learning a different topic e.g. the species names and how they sound, unfortunately.
", "votes": 1, "replies": [] }, { "id": 1082463, "author_name": "", "author_url": "", "post_date": "2020-11-17T23:17:52.433000", "content": "The evaluation is based only on predicting species presence (not songtype), but as indicated by songtype_id some species have two different songs in the training data. In these cases both songtypes are common signals from the species. Songtype 1 indicates the most common song as determined by a biologist on our team, songtype 4 just indicates another common song variation.
I understand there is less interesting biology to learn without species names, but currently we think preventing extra data collection is priority, and are interested in solutions that can generalize well to species outside our dataset
", "votes": 9, "replies": [] }, { "id": 1088044, "author_name": "tropicbird", "author_url": "", "post_date": "2020-11-23T09:11:18.707000", "content": "@jacklebien Thank you for your explanation! I understood the reason why the species names are hidden. However, it is still allowed to use external audio data to train our models, right? I assume some participants (e.g. specialists in biology) might be able to find out the name of species. Then, they could collect the specific additional audio data easily, which the competition host does not want. Also, it might be unfair for the other participants.
\nSo, my question is, do you have any rule (or requirements) if participants find out the name of species? Thank you!
", "votes": 2, "replies": [] }, { "id": 1088047, "author_name": "Ali Abdin", "author_url": "", "post_date": "2020-11-23T09:15:41.413000", "content": "From the rules:
\n\n", "votes": 0, "replies": [] }, { "id": 1088543, "author_name": "Julia Elliott", "author_url": "", "post_date": "2020-11-23T17:59:51.490000", "content": "External Data:
\n
\n You may use data other than the Competition Data (“External Data”) to develop and test your Submissions. However, you will ensure the External Data is publicly available to use by all participants of the Competition for purposes of the competition at no cost to the other participants. The ability to use External Data under this Section 7.C (External Data) does not limit you other obligations under these Competition Rules, including but not limited to Section 11 (Winners Obligations).
@hdsk38 Yes, it is permitted to use External data, as allowed by the rules. So if you were to decipher the species and/or find additional external data, it would be permitted. Hand-labeling, however, is not permitted, including if you incorporate hand-labeling within your model's training.
", "votes": 0, "replies": [] }, { "id": 1088571, "author_name": "fffrrt", "author_url": "", "post_date": "2020-11-23T18:42:33.043000", "content": "Can you clarify if hand-labeling and/or editing of training data is permitted?
\nIt is unclear for me if your post refers to total ban on hand-labeling or it just refers to hypothetical situation where someone hand-labels test data and adds it to model training data. In previous posts only hand-labeling test data was explicitly banned.
", "votes": 0, "replies": [] }, { "id": 1143265, "author_name": "CPMP", "author_url": "", "post_date": "2021-01-07T20:18:31.447000", "content": "@jacklebien you wrote:
\n\n\nWe chose to hide the species names to discourage collection of more training data from external sources and encourage solutions for limited training data.
\n
and
\n\n\nwe think preventing extra data collection is priority
\n
Then why is external data authorized as @juliaelliott reminds us?
", "votes": 1, "replies": [] }, { "id": 1143285, "author_name": "beluga", "author_url": "", "post_date": "2021-01-07T20:32:16.617000", "content": "I think the rules do not force the intent of the hosts…
\nFrom the rules:
\n\n\n\n
\n- COMPETITION ENTRY…
\n
\n Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.
I will be nice and Kaggley (as @inversion asked in the other thread) and I won't try to decipher the data and collect more and I obviously won't try to hand label the test dataset. However I might try to fix the training labels manually…
", "votes": 2, "replies": [] }, { "id": 1143421, "author_name": "Julia Elliott", "author_url": "", "post_date": "2021-01-07T22:36:09.757000", "content": "Hi, all. Sorry that these responses have caused some confusion. I have fully clarified with the host, and we are in agreement that: External data is allowed and taking actions to include external data — including that which makes use of hand-labeled species on the training data — is fully permitted, per the competition rules.
", "votes": 4, "replies": [] } ] } ], "raw_markdown_by_id": { "1082386": "* species_id - unique identifier for species\n* songtype_id - unique identifier for songtype\n\nMaybe I am just missing something but I can't find the taxonomy for the species only numeric ids.\nCollecting external data (e.g. Xeno Canto) would be easier if we knew the species in advance.\n\n", "1200197": "Here is the list of the species: https://www.sciencedirect.com/science/article/pii/S1574954120300637", "1119904": "@jacklebien: Could you reveal the names of the species once the competition is over, please? I'd like to see the birds / frogs that make these sounds.", "1082445": "Hello! We chose to hide the species names to discourage collection of more training data from external sources and encourage solutions for limited training data. \n\nSolutions requiring minimal training data will accelerate the development of eco-acoustic monitoring systems, and especially help with rare species where training data is difficult to find." } }