{ "id": 501241, "title": "Early Stopping?", "url": "/competitions/birdclef-2024/discussion/501241", "author_name": "Yacine Bouaouni", "post_date": "2024-05-08T15:22:48.063000", "votes": 1, "comment_count": 7, "views": 0, "content": "

Hello,

\n

Based on these two plots for train loss and validation metric, would you put an early stopping (before 10 epochs) because the validation score is decreasing or let the training finish since it converges after 40 epochs.
\n
\n

", "messages": [ { "id": 2801508, "postDate": "2024-05-08T16:39:31.040Z", "content": "

I don't use early stopping.

", "rawMarkdown": "I don't use early stopping.", "votes": 3, "replies": [ { "id": 2802108, "postDate": "2024-05-08T21:42:53.533Z", "content": "

That is extremely interesting! Could I ask how many epochs you use?

", "rawMarkdown": "That is extremely interesting! Could I ask how many epochs you use?", "votes": 1, "replies": [ { "id": 2803651, "postDate": "2024-05-09T15:32:09.763Z", "content": "

The number of epochs alone is not meaningful. You'd have to consider the model used, the training data, the learning rate, the batch size, etc.

\n

My point is that using early stopping is brittle. It is better to use the last checkpoint and tune hyper parameters to improve it than to rely on overfitting the validation data via early stopping.

", "rawMarkdown": "The number of epochs alone is not meaningful. You'd have to consider the model used, the training data, the learning rate, the batch size, etc.\n\nMy point is that using early stopping is brittle. It is better to use the last checkpoint and tune hyper parameters to improve it than to rely on overfitting the validation data via early stopping.", "votes": 2 }, { "id": 2803722, "postDate": "2024-05-09T15:50:27.327Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2803759, "postDate": "2024-05-09T15:58:59.537Z", "content": "

I prefer not to share specifics of what I do. But you can look at past competition top solutions to get an idea of what worked for them.

", "rawMarkdown": "I prefer not to share specifics of what I do. But you can look at past competition top solutions to get an idea of what worked for them." }, { "id": 2803865, "postDate": "2024-05-09T16:55:32.043Z", "rawMarkdown": "", "isDeleted": true } ] } ] }, { "id": 2801319, "postDate": "2024-05-08T15:22:48.063Z", "content": "

Hello,

\n

Based on these two plots for train loss and validation metric, would you put an early stopping (before 10 epochs) because the validation score is decreasing or let the training finish since it converges after 40 epochs.
\n
\n

", "rawMarkdown": "Hello,\n\nBased on these two plots for train loss and validation metric, would you put an early stopping (before 10 epochs) because the validation score is decreasing or let the training finish since it converges after 40 epochs.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3775861%2Fa848960fdf93f18f79b68d7f8bb93973%2Ftrain_loss_plot.png?generation=1715181456938032&alt=media)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3775861%2F6692f17d8cf123e908918c90e4c39631%2Fvalidation_score_plot.png?generation=1715181488957771&alt=media)", "votes": 1 }, { "id": 2801894, "postDate": "2024-05-08T19:28:23.197Z", "content": "

This is definitely overfitting, use the validation loss too in order to see the behavior. What optimizer, lr scheduler and settings do you use?

", "rawMarkdown": "This is definitely overfitting, use the validation loss too in order to see the behavior. What optimizer, lr scheduler and settings do you use?" } ], "comments": [ { "id": 2801508, "author_name": "CPMP", "author_url": "", "post_date": "2024-05-08T16:39:31.040000", "content": "

I don't use early stopping.

", "votes": 3, "replies": [ { "id": 2802108, "author_name": "Cody_Null", "author_url": "", "post_date": "2024-05-08T21:42:53.533000", "content": "

That is extremely interesting! Could I ask how many epochs you use?

", "votes": 1, "replies": [ { "id": 2803651, "author_name": "CPMP", "author_url": "", "post_date": "2024-05-09T15:32:09.763000", "content": "

The number of epochs alone is not meaningful. You'd have to consider the model used, the training data, the learning rate, the batch size, etc.

\n

My point is that using early stopping is brittle. It is better to use the last checkpoint and tune hyper parameters to improve it than to rely on overfitting the validation data via early stopping.

", "votes": 2, "replies": [] }, { "id": 2803722, "author_name": "", "author_url": "", "post_date": "2024-05-09T15:50:27.327000", "content": "", "votes": 0, "replies": [] }, { "id": 2803759, "author_name": "CPMP", "author_url": "", "post_date": "2024-05-09T15:58:59.537000", "content": "

I prefer not to share specifics of what I do. But you can look at past competition top solutions to get an idea of what worked for them.

", "votes": 0, "replies": [] }, { "id": 2803865, "author_name": "", "author_url": "", "post_date": "2024-05-09T16:55:32.043000", "content": "", "votes": 0, "replies": [] } ] } ] }, { "id": 2801894, "author_name": "SJ", "author_url": "", "post_date": "2024-05-08T19:28:23.197000", "content": "

This is definitely overfitting, use the validation loss too in order to see the behavior. What optimizer, lr scheduler and settings do you use?

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2801508": "I don't use early stopping.", "2801319": "Hello,\n\nBased on these two plots for train loss and validation metric, would you put an early stopping (before 10 epochs) because the validation score is decreasing or let the training finish since it converges after 40 epochs.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3775861%2Fa848960fdf93f18f79b68d7f8bb93973%2Ftrain_loss_plot.png?generation=1715181456938032&alt=media)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3775861%2F6692f17d8cf123e908918c90e4c39631%2Fvalidation_score_plot.png?generation=1715181488957771&alt=media)", "2801894": "This is definitely overfitting, use the validation loss too in order to see the behavior. What optimizer, lr scheduler and settings do you use?" } }