I just submitted one model from my solution to previous competition to get a baseline, and I noticed several changes that are worth investigating.
\nLast year metric was f1 score by sample (row wise). This year it is f1 score macro, which is column wise. I think it is a positive change, but tuning done for last year metric may not be best now.
Last year we had some soundscape annotated every 5 second which we could use as validation data. This year there aren't. Defining a good validation setting will probably be key this year. Unless we rely on public LB of course. I tend to avoid the latter, but that's just my opinion.
The number of scored species is way smaller than last year. Note sure yet about the consequences.
All recordings are from a single location (Hawaii). This makes geo based work way less relevant. Still Hawaii has different biotopes, but I don't know if location data can help.
I just submitted one model from my solution to previous competition to get a baseline, and I noticed several changes that are worth investigating.
\nLast year metric was f1 score by sample (row wise). This year it is f1 score macro, which is column wise. I think it is a positive change, but tuning done for last year metric may not be best now.
Last year we had some soundscape annotated every 5 second which we could use as validation data. This year there aren't. Defining a good validation setting will probably be key this year. Unless we rely on public LB of course. I tend to avoid the latter, but that's just my opinion.
The number of scored species is way smaller than last year. Note sure yet about the consequences.
All recordings are from a single location (Hawaii). This makes geo based work way less relevant. Still Hawaii has different biotopes, but I don't know if location data can help.
I think that the validation per audio file (even you can cut each audio per 5s frames) seem to give rather robust validation (without leaking training data), I used it in https://www.kaggle.com/jirkaborovec/birdclef-eda-multi-class-flash-effnet
", "rawMarkdown": "I think that the validation per audio file (even you can cut each audio per 5s frames) seem to give rather robust validation (without leaking training data), I used it in https://www.kaggle.com/jirkaborovec/birdclef-eda-multi-class-flash-effnet", "votes": 1 }, { "id": 1701327, "postDate": "2022-02-22T17:15:28.117Z", "content": "yes, there were tips to use geography … rather possible combinations of birds in one sound recording
", "rawMarkdown": "yes, there were [tips to use geography](https://www.kaggle.com/c/birdclef-2022/discussion/307903) ... rather possible combinations of birds in one sound recording", "votes": 1, "replies": [ { "id": 1701447, "postDate": "2022-02-22T19:39:14.537Z", "content": "This has been use din previous competition, see the write ups. I am questioning the relevance here still, but we'll see.
", "rawMarkdown": "This has been use din previous competition, see the write ups. I am questioning the relevance here still, but we'll see.", "votes": 1 } ] }, { "id": 1717374, "postDate": "2022-03-09T21:56:31.763Z", "content": "regarding the 5s frames, I was asking similar in this discussion: Is bird presented all the time in training recordings?
", "rawMarkdown": "regarding the 5s frames, I was asking similar in this discussion: [Is bird presented all the time in training recordings?](https://www.kaggle.com/c/birdclef-2022/discussion/308861)", "votes": -1, "replies": [ { "id": 1717966, "postDate": "2022-03-10T11:43:04.667Z", "content": "This is a different topic. In previous competition training data was not labelled by 5 second segements either.
", "rawMarkdown": "This is a different topic. In previous competition training data was not labelled by 5 second segements either.", "votes": 1 }, { "id": 1717992, "postDate": "2022-03-10T12:14:11.717Z", "content": "true, just downloaded the dataset…
", "rawMarkdown": "true, just downloaded the dataset..." } ] }, { "id": 1790514, "postDate": "2022-05-15T03:48:03.223Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1717373, "author_name": "Jirka", "author_url": "", "post_date": "2022-03-09T21:55:15.300000", "content": "I think that the validation per audio file (even you can cut each audio per 5s frames) seem to give rather robust validation (without leaking training data), I used it in https://www.kaggle.com/jirkaborovec/birdclef-eda-multi-class-flash-effnet
", "votes": 1, "replies": [] }, { "id": 1701327, "author_name": "Pavel Orlov", "author_url": "", "post_date": "2022-02-22T17:15:28.117000", "content": "yes, there were tips to use geography … rather possible combinations of birds in one sound recording
", "votes": 1, "replies": [ { "id": 1701447, "author_name": "CPMP", "author_url": "", "post_date": "2022-02-22T19:39:14.537000", "content": "This has been use din previous competition, see the write ups. I am questioning the relevance here still, but we'll see.
", "votes": 1, "replies": [] } ] }, { "id": 1717374, "author_name": "Jirka", "author_url": "", "post_date": "2022-03-09T21:56:31.763000", "content": "regarding the 5s frames, I was asking similar in this discussion: Is bird presented all the time in training recordings?
", "votes": -1, "replies": [ { "id": 1717966, "author_name": "CPMP", "author_url": "", "post_date": "2022-03-10T11:43:04.667000", "content": "This is a different topic. In previous competition training data was not labelled by 5 second segements either.
", "votes": 1, "replies": [] }, { "id": 1717992, "author_name": "Jirka", "author_url": "", "post_date": "2022-03-10T12:14:11.717000", "content": "true, just downloaded the dataset…
", "votes": 0, "replies": [] } ] }, { "id": 1790514, "author_name": "", "author_url": "", "post_date": "2022-05-15T03:48:03.223000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1700920": "I just submitted one model from my solution to previous competition to get a baseline, and I noticed several changes that are worth investigating.\n\n1. Last year metric was f1 score by sample (row wise). This year it is f1 score macro, which is column wise. I think it is a positive change, but tuning done for last year metric may not be best now. \n\n2. Last year we had some soundscape annotated every 5 second which we could use as validation data. This year there aren't. Defining a good validation setting will probably be key this year. Unless we rely on public LB of course. I tend to avoid the latter, but that's just my opinion.\n\n3. The number of scored species is way smaller than last year. Note sure yet about the consequences.\n\n4. All recordings are from a single location (Hawaii). This makes geo based work way less relevant. Still Hawaii has different biotopes, but I don't know if location data can help.\n\n", "1717373": "I think that the validation per audio file (even you can cut each audio per 5s frames) seem to give rather robust validation (without leaking training data), I used it in https://www.kaggle.com/jirkaborovec/birdclef-eda-multi-class-flash-effnet", "1701327": "yes, there were [tips to use geography](https://www.kaggle.com/c/birdclef-2022/discussion/307903) ... rather possible combinations of birds in one sound recording", "1717374": "regarding the 5s frames, I was asking similar in this discussion: [Is bird presented all the time in training recordings?](https://www.kaggle.com/c/birdclef-2022/discussion/308861)", "1790514": "" } }