I'm a Bird competition novice. I'm not sure what the previous competition was like yet, so I'd greatly appreciate it if anyone who knows can compare and let me know.
\nCV can be up to 0.99, but I still can't believe that it's really possible. Even considering the leak, it's about 0.9~, but will it be this high in ROC-AUC even if the model is not very good? Those who have pre-trained are curious about how ROC-AUC is in the previous competition data, and if anyone has learned the data from this year's competition with the metric of the previous competition, I'm curious about the figures.
\nI think one of the reasons why it's low in LB is because of the secondary label. I don't know much about birdsong, but in some samples, you can hear more than three different species of birdsong at the same time. And in some samples, the sound can be heard very small from very far away. (The model has managed to catch these parts as well, surprisingly.) Do we have to match all of these birdsong?
", "messages": [ { "id": "2775952", "postDate": "04/25/2024 22:52:29", "content": "I'm a Bird competition novice. I'm not sure what the previous competition was like yet, so I'd greatly appreciate it if anyone who knows can compare and let me know.
\nCV can be up to 0.99, but I still can't believe that it's really possible. Even considering the leak, it's about 0.9~, but will it be this high in ROC-AUC even if the model is not very good? Those who have pre-trained are curious about how ROC-AUC is in the previous competition data, and if anyone has learned the data from this year's competition with the metric of the previous competition, I'm curious about the figures.
\nI think one of the reasons why it's low in LB is because of the secondary label. I don't know much about birdsong, but in some samples, you can hear more than three different species of birdsong at the same time. And in some samples, the sound can be heard very small from very far away. (The model has managed to catch these parts as well, surprisingly.) Do we have to match all of these birdsong?
", "rawMarkdown": "I'm a Bird competition novice. I'm not sure what the previous competition was like yet, so I'd greatly appreciate it if anyone who knows can compare and let me know.\n\nCV can be up to 0.99, but I still can't believe that it's really possible. Even considering the leak, it's about 0.9~, but will it be this high in ROC-AUC even if the model is not very good? Those who have pre-trained are curious about how ROC-AUC is in the previous competition data, and if anyone has learned the data from this year's competition with the metric of the previous competition, I'm curious about the figures.\n\nI think one of the reasons why it's low in LB is because of the secondary label. I don't know much about birdsong, but in some samples, you can hear more than three different species of birdsong at the same time. And in some samples, the sound can be heard very small from very far away. (The model has managed to catch these parts as well, surprisingly.) Do we have to match all of these birdsong?", "votes": null }, { "id": "2775981", "postDate": "04/25/2024 23:58:17", "content": "\n\nEven considering the leak,
\n
Which leak do you consider?
", "rawMarkdown": "> Even considering the leak, \n\nWhich leak do you consider?", "votes": null }, { "id": "2775997", "postDate": "04/26/2024 00:11:09", "content": "The same audio that was shared in the thread was being used over and over again. Not technically a leak, but audio that was recorded over and over again in the same place. Surprisingly, when I removed duplicate audio, my CV deteriorated.
", "rawMarkdown": "The same audio that was shared in the thread was being used over and over again. Not technically a leak, but audio that was recorded over and over again in the same place. Surprisingly, when I removed duplicate audio, my CV deteriorated.", "votes": null }, { "id": "2776166", "postDate": "04/26/2024 03:13:02", "content": "\"What is the main reason for the difference between CV and LB?\" For training, specific recordings of a particular bird are chosen. When this bird is nearby and easily audible (and this is what you use for CV). However, the test is conducted in the 'wild' environment, where there are many different noises and other sounds around. The bird may be somewhere far away. (and it is on these recordings that the LB metric is calculated).
", "rawMarkdown": "\"What is the main reason for the difference between CV and LB?\" For training, specific recordings of a particular bird are chosen. When this bird is nearby and easily audible (and this is what you use for CV). However, the test is conducted in the 'wild' environment, where there are many different noises and other sounds around. The bird may be somewhere far away. (and it is on these recordings that the LB metric is calculated).", "votes": null }, { "id": "2779968", "postDate": "04/28/2024 00:35:01", "content": "I also consider the leakage problem as many people use pre-train models in this competition, and I wonder if some data was used in the pre-training stage before, since there are many duplicate species (14 common birds between 2023 and 2024, might be more from previous years). If so the pretrained model might overfitting and result in a very high CV.
", "rawMarkdown": "I also consider the leakage problem as many people use pre-train models in this competition, and I wonder if some data was used in the pre-training stage before, since there are many duplicate species (14 common birds between 2023 and 2024, might be more from previous years). If so the pretrained model might overfitting and result in a very high CV.", "votes": null }, { "id": "2780136", "postDate": "04/28/2024 04:32:56", "content": "Which audio was shared to be repeated over and over? I missed that.
", "rawMarkdown": "Which audio was shared to be repeated over and over? I missed that.", "votes": null }, { "id": "2780260", "postDate": "04/28/2024 05:34:05", "content": "Our team is not using the previous data yet, but the processing method of the previous data will also be a key point
", "rawMarkdown": "Our team is not using the previous data yet, but the processing method of the previous data will also be a key point", "votes": null }, { "id": "2780262", "postDate": "04/28/2024 05:35:22", "content": "https://www.kaggle.com/code/robbynevels/bc24-duplicate-audio-files/
\nI don't really remember if there's anything else besides this, but it didn't work that well.
Thanks. I saw that one, but it is not one audio repeated many times.
", "rawMarkdown": "Thanks. I saw that one, but it is not one audio repeated many times.", "votes": null }, { "id": "2780283", "postDate": "04/28/2024 05:53:36", "content": "I am not using previous year data either at this point.
", "rawMarkdown": "I am not using previous year data either at this point.", "votes": null }, { "id": "2780317", "postDate": "04/28/2024 06:20:16", "content": "I was a little puzzled by your reaction from the beginning, but now I know. I'm sorry if the expression was weird due to my lack of English skills.
", "rawMarkdown": "I was a little puzzled by your reaction from the beginning, but now I know. I'm sorry if the expression was weird due to my lack of English skills.", "votes": null }, { "id": "2780336", "postDate": "04/28/2024 06:28:32", "content": "No pb. I thought you were discussing these duplicates but I preferred to check.
\nI am not a native english speaker either, and sometimes I also have issues to get my ideas out in english.
", "rawMarkdown": "No pb. I thought you were discussing these duplicates but I preferred to check.\n\nI am not a native english speaker either, and sometimes I also have issues to get my ideas out in english.", "votes": null } ], "comments": [ { "id": 2775981, "author_name": "cpmpml", "author_url": "", "post_date": "04/25/2024 23:58:17", "content": "\n\nEven considering the leak,
\n
Which leak do you consider?
", "votes": null, "replies": [ { "id": 2775997, "author_name": "ajobseeker", "author_url": "", "post_date": "04/26/2024 00:11:09", "content": "The same audio that was shared in the thread was being used over and over again. Not technically a leak, but audio that was recorded over and over again in the same place. Surprisingly, when I removed duplicate audio, my CV deteriorated.
", "votes": null, "replies": [ { "id": 2780136, "author_name": "cpmpml", "author_url": "", "post_date": "04/28/2024 04:32:56", "content": "Which audio was shared to be repeated over and over? I missed that.
", "votes": null, "replies": [ { "id": 2780262, "author_name": "ajobseeker", "author_url": "", "post_date": "04/28/2024 05:35:22", "content": "https://www.kaggle.com/code/robbynevels/bc24-duplicate-audio-files/
\nI don't really remember if there's anything else besides this, but it didn't work that well.
Thanks. I saw that one, but it is not one audio repeated many times.
", "votes": null, "replies": [ { "id": 2780317, "author_name": "ajobseeker", "author_url": "", "post_date": "04/28/2024 06:20:16", "content": "I was a little puzzled by your reaction from the beginning, but now I know. I'm sorry if the expression was weird due to my lack of English skills.
", "votes": null, "replies": [ { "id": 2780336, "author_name": "cpmpml", "author_url": "", "post_date": "04/28/2024 06:28:32", "content": "No pb. I thought you were discussing these duplicates but I preferred to check.
\nI am not a native english speaker either, and sometimes I also have issues to get my ideas out in english.
", "votes": null, "replies": [] } ] } ] } ] } ] } ] }, { "id": 2779968, "author_name": "leonshangguan", "author_url": "", "post_date": "04/28/2024 00:35:01", "content": "I also consider the leakage problem as many people use pre-train models in this competition, and I wonder if some data was used in the pre-training stage before, since there are many duplicate species (14 common birds between 2023 and 2024, might be more from previous years). If so the pretrained model might overfitting and result in a very high CV.
", "votes": null, "replies": [ { "id": 2780260, "author_name": "ajobseeker", "author_url": "", "post_date": "04/28/2024 05:34:05", "content": "Our team is not using the previous data yet, but the processing method of the previous data will also be a key point
", "votes": null, "replies": [ { "id": 2780283, "author_name": "cpmpml", "author_url": "", "post_date": "04/28/2024 05:53:36", "content": "I am not using previous year data either at this point.
", "votes": null, "replies": [] } ] } ] } ] }, { "id": 2776166, "author_name": "sapr3s", "author_url": "", "post_date": "04/26/2024 03:13:02", "content": "\"What is the main reason for the difference between CV and LB?\" For training, specific recordings of a particular bird are chosen. When this bird is nearby and easily audible (and this is what you use for CV). However, the test is conducted in the 'wild' environment, where there are many different noises and other sounds around. The bird may be somewhere far away. (and it is on these recordings that the LB metric is calculated).
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2775952": "I'm a Bird competition novice. I'm not sure what the previous competition was like yet, so I'd greatly appreciate it if anyone who knows can compare and let me know.\n\nCV can be up to 0.99, but I still can't believe that it's really possible. Even considering the leak, it's about 0.9~, but will it be this high in ROC-AUC even if the model is not very good? Those who have pre-trained are curious about how ROC-AUC is in the previous competition data, and if anyone has learned the data from this year's competition with the metric of the previous competition, I'm curious about the figures.\n\nI think one of the reasons why it's low in LB is because of the secondary label. I don't know much about birdsong, but in some samples, you can hear more than three different species of birdsong at the same time. And in some samples, the sound can be heard very small from very far away. (The model has managed to catch these parts as well, surprisingly.) Do we have to match all of these birdsong?", "2775981": "> Even considering the leak, \n\nWhich leak do you consider?", "2775997": "The same audio that was shared in the thread was being used over and over again. Not technically a leak, but audio that was recorded over and over again in the same place. Surprisingly, when I removed duplicate audio, my CV deteriorated.", "2776166": "\"What is the main reason for the difference between CV and LB?\" For training, specific recordings of a particular bird are chosen. When this bird is nearby and easily audible (and this is what you use for CV). However, the test is conducted in the 'wild' environment, where there are many different noises and other sounds around. The bird may be somewhere far away. (and it is on these recordings that the LB metric is calculated).", "2779968": "I also consider the leakage problem as many people use pre-train models in this competition, and I wonder if some data was used in the pre-training stage before, since there are many duplicate species (14 common birds between 2023 and 2024, might be more from previous years). If so the pretrained model might overfitting and result in a very high CV.", "2780136": "Which audio was shared to be repeated over and over? I missed that.", "2780260": "Our team is not using the previous data yet, but the processing method of the previous data will also be a key point", "2780262": "https://www.kaggle.com/code/robbynevels/bc24-duplicate-audio-files/\nI don't really remember if there's anything else besides this, but it didn't work that well.", "2780273": "Thanks. I saw that one, but it is not one audio repeated many times.", "2780283": "I am not using previous year data either at this point.", "2780317": "I was a little puzzled by your reaction from the beginning, but now I know. I'm sorry if the expression was weird due to my lack of English skills.", "2780336": "No pb. I thought you were discussing these duplicates but I preferred to check.\n\nI am not a native english speaker either, and sometimes I also have issues to get my ideas out in english." }, "source": "meta" }