Is there anyone tried to use categorical cross entropy loss during training rather than binary cross entropy and i tried to use it I found that the cv score became really good around 94 auc but the leaderboard became 64 auc I don’t know why
", "messages": [ { "id": 3178585, "postDate": "2025-04-14T11:01:27.343Z", "content": "Is there anyone tried to use categorical cross entropy loss during training rather than binary cross entropy and i tried to use it I found that the cv score became really good around 94 auc but the leaderboard became 64 auc I don’t know why
", "rawMarkdown": "Is there anyone tried to use categorical cross entropy loss during training rather than binary cross entropy and i tried to use it I found that the cv score became really good around 94 auc but the leaderboard became 64 auc I don’t know why ", "votes": 3 }, { "id": 3186545, "postDate": "2025-04-24T20:22:01.650Z", "content": "can you explain cross entropy?
", "rawMarkdown": "can you explain cross entropy?", "votes": 1, "replies": [ { "id": 3201028, "postDate": "2025-05-13T11:03:43.800Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 3178727, "postDate": "2025-04-14T14:23:14.393Z", "content": "This is a multi label problem (that is every segment can have multiple species sounds) rather than multi class. BCE Loss with sigmoid should work better. Categorical cross entropy would work better for multi class problem.
", "rawMarkdown": "This is a multi label problem (that is every segment can have multiple species sounds) rather than multi class. BCE Loss with sigmoid should work better. Categorical cross entropy would work better for multi class problem.", "votes": 1, "replies": [ { "id": 3178851, "postDate": "2025-04-14T17:30:09.263Z", "content": "that's right but the winner in the previous birdclef comp_ tried this way during training and the task was also multilabel classification task and his results was very good compared to use binary cross entropy loss during training i thought it will work here specially when i saw that the cv score became better
", "rawMarkdown": "that's right but the winner in the previous birdclef comp_ tried this way during training and the task was also multilabel classification task and his results was very good compared to use binary cross entropy loss during training i thought it will work here specially when i saw that the cv score became better ", "votes": 1, "replies": [ { "id": 3178878, "postDate": "2025-04-14T18:08:28.817Z", "content": "I just read their solution, quite elaborate - they did sigmoid for inference and CE + softmax for training .
\nThey also say -
\n\nUsing the sigmoid with CE-trained models leads to the predictions being noisy, so min() just lowers uncertain predictions.
\n
Also I noticed last year ~7% of data had secondary labels, this year we have ~9% data with secondary labels.
", "rawMarkdown": "I just read [their solution](https://www.kaggle.com/competitions/birdclef-2024/discussion/512197), quite elaborate - they did sigmoid for inference and CE + softmax for training . \nThey also say - \n>Using the sigmoid with CE-trained models leads to the predictions being noisy, so min() just lowers uncertain predictions.\n\nAlso I noticed last year ~7% of data had secondary labels, this year we have ~9% data with secondary labels.\n", "votes": 2 } ] } ] }, { "id": 3178857, "postDate": "2025-04-14T17:37:58.727Z", "content": "I tried using cross entropy loss and for inference I used BCE as suggested by last year's winning team's approach.
\nI got CV 10 epochs = 0.9662 LB =0.768
", "rawMarkdown": "I tried using cross entropy loss and for inference I used BCE as suggested by last year's winning team's approach.\n\nI got CV 10 epochs = 0.9662\tLB =0.768\n", "replies": [ { "id": 3179027, "postDate": "2025-04-14T23:09:16.243Z", "content": "Did you face any difficulties during creating this approach
", "rawMarkdown": "Did you face any difficulties during creating this approach ", "votes": -1 }, { "id": 3179031, "postDate": "2025-04-14T23:21:10.793Z", "content": "Did you ensemble 5 folds or took just one of them
", "rawMarkdown": "Did you ensemble 5 folds or took just one of them ", "replies": [ { "id": 3179773, "postDate": "2025-04-15T17:04:58.397Z", "content": "I didn't face any difficulties recreating this approach, and yes I used 5 folds with effecientnet B0
", "rawMarkdown": "I didn't face any difficulties recreating this approach, and yes I used 5 folds with effecientnet B0", "votes": 1 }, { "id": 3180899, "postDate": "2025-04-17T09:07:49.410Z", "content": "Hi there. I am curious to know how you are training 5 folds. Training for 15 epochs on one fold using efficientnet B0 takes about 5hrs on GPU. So training 5 folds won't be possible in a single notebook (or is it?). Are you training in a single notebook or multiple notebooks?
", "rawMarkdown": "Hi there. I am curious to know how you are training 5 folds. Training for 15 epochs on one fold using efficientnet B0 takes about 5hrs on GPU. So training 5 folds won't be possible in a single notebook (or is it?). Are you training in a single notebook or multiple notebooks?", "votes": 1 }, { "id": 3181098, "postDate": "2025-04-17T13:13:57.983Z", "content": "i think they made it in different notebooks and take the output of each notebook and put them in the inference notebook
", "rawMarkdown": "i think they made it in different notebooks and take the output of each notebook and put them in the inference notebook ", "votes": 1 } ] } ] }, { "id": 3201032, "postDate": "2025-05-13T11:11:11.783Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 3186545, "author_name": "", "author_url": "", "post_date": "2025-04-24T20:22:01.650000", "content": "can you explain cross entropy?
", "votes": 1, "replies": [ { "id": 3201028, "author_name": "", "author_url": "", "post_date": "2025-05-13T11:03:43.800000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 3178727, "author_name": "RB", "author_url": "", "post_date": "2025-04-14T14:23:14.393000", "content": "This is a multi label problem (that is every segment can have multiple species sounds) rather than multi class. BCE Loss with sigmoid should work better. Categorical cross entropy would work better for multi class problem.
", "votes": 1, "replies": [ { "id": 3178851, "author_name": "RIYADH HAITHAM", "author_url": "", "post_date": "2025-04-14T17:30:09.263000", "content": "that's right but the winner in the previous birdclef comp_ tried this way during training and the task was also multilabel classification task and his results was very good compared to use binary cross entropy loss during training i thought it will work here specially when i saw that the cv score became better
", "votes": 1, "replies": [ { "id": 3178878, "author_name": "RB", "author_url": "", "post_date": "2025-04-14T18:08:28.817000", "content": "I just read their solution, quite elaborate - they did sigmoid for inference and CE + softmax for training .
\nThey also say -
\n\nUsing the sigmoid with CE-trained models leads to the predictions being noisy, so min() just lowers uncertain predictions.
\n
Also I noticed last year ~7% of data had secondary labels, this year we have ~9% data with secondary labels.
", "votes": 2, "replies": [] } ] } ] }, { "id": 3178857, "author_name": "Devasy Patel23", "author_url": "", "post_date": "2025-04-14T17:37:58.727000", "content": "I tried using cross entropy loss and for inference I used BCE as suggested by last year's winning team's approach.
\nI got CV 10 epochs = 0.9662 LB =0.768
", "votes": 0, "replies": [ { "id": 3179027, "author_name": "RIYADH HAITHAM", "author_url": "", "post_date": "2025-04-14T23:09:16.243000", "content": "Did you face any difficulties during creating this approach
", "votes": -1, "replies": [] }, { "id": 3179031, "author_name": "RIYADH HAITHAM", "author_url": "", "post_date": "2025-04-14T23:21:10.793000", "content": "Did you ensemble 5 folds or took just one of them
", "votes": 0, "replies": [ { "id": 3179773, "author_name": "Devasy Patel23", "author_url": "", "post_date": "2025-04-15T17:04:58.397000", "content": "I didn't face any difficulties recreating this approach, and yes I used 5 folds with effecientnet B0
", "votes": 1, "replies": [] }, { "id": 3180899, "author_name": "Abhisek Dash", "author_url": "", "post_date": "2025-04-17T09:07:49.410000", "content": "Hi there. I am curious to know how you are training 5 folds. Training for 15 epochs on one fold using efficientnet B0 takes about 5hrs on GPU. So training 5 folds won't be possible in a single notebook (or is it?). Are you training in a single notebook or multiple notebooks?
", "votes": 1, "replies": [] }, { "id": 3181098, "author_name": "RIYADH HAITHAM", "author_url": "", "post_date": "2025-04-17T13:13:57.983000", "content": "i think they made it in different notebooks and take the output of each notebook and put them in the inference notebook
", "votes": 1, "replies": [] } ] } ] }, { "id": 3201032, "author_name": "", "author_url": "", "post_date": "2025-05-13T11:11:11.783000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "3178585": "Is there anyone tried to use categorical cross entropy loss during training rather than binary cross entropy and i tried to use it I found that the cv score became really good around 94 auc but the leaderboard became 64 auc I don’t know why ", "3186545": "can you explain cross entropy?", "3178727": "This is a multi label problem (that is every segment can have multiple species sounds) rather than multi class. BCE Loss with sigmoid should work better. Categorical cross entropy would work better for multi class problem.", "3178857": "I tried using cross entropy loss and for inference I used BCE as suggested by last year's winning team's approach.\n\nI got CV 10 epochs = 0.9662\tLB =0.768\n", "3201032": "" } }