They should be the same, right? I got confused.
", "messages": [ { "id": "745493", "postDate": "02/13/2020 21:31:23", "content": "They should be the same, right? I got confused.
", "rawMarkdown": "They should be the same, right? I got confused.", "votes": null }, { "id": "745621", "postDate": "02/14/2020 02:19:05", "content": "To my understanding, they are different. Early stopping gets a model, some steps after the monitoring score doesn't improve. So the model is not a best validation scoring model. And, if you save models based on validation, it is not always the last epoch model. Of course, you can combine these two.
\n\nIn my case of this competition, I don't use early stopping, but save models based on validation.\nThis is because... when I got unexpectedly high score in the early epochs, if using early stopping, the model stop training soon after the high validation score even when it is not fully trained by train dataset.
\n\nPlease correct if I am wrong🙏
", "rawMarkdown": "To my understanding, they are different. Early stopping gets a model, some steps after the monitoring score doesn't improve. So the model is not a best validation scoring model. And, if you save models based on validation, it is not always the last epoch model. Of course, you can combine these two.\n\nIn my case of this competition, I don't use early stopping, but save models based on validation.\nThis is because... when I got unexpectedly high score in the early epochs, if using early stopping, the model stop training soon after the high validation score even when it is not fully trained by train dataset.\n\nPlease correct if I am wrong🙏", "votes": null }, { "id": "745632", "postDate": "02/14/2020 02:50:03", "content": "Yeah that's what I was thinking too. Thanks! @inoueu1 I am asking this because I found that my codes for this competition don't seem to work perfectly by saving on validation, so I will probably try early stopping.
", "rawMarkdown": "Yeah that's what I was thinking too. Thanks! @inoueu1 I am asking this because I found that my codes for this competition don't seem to work perfectly by saving on validation, so I will probably try early stopping.", "votes": null }, { "id": "748633", "postDate": "02/17/2020 19:11:37", "content": "Their main difference is that early stopping is \"stop if learning isn't happening\" whereas saving is \"given training cycle get best validation-scoring model\". It is possible that they may give the same model, but since they both do require some external tuning (what to assess on? How many epochs to wait till it actually stops? (this is referred as patience usually), etc so they can certainly provide different results. What I like to do is to do saving based on validation at all times to get the best valid scoring ckpt, while applying early stopping on top of it if I am time-restricted, or cases where it could be that the model converged before even 1/3 of epochs specified has reached (hence to prevent wasting time). Hope that gives you a better idea about the two :D
", "rawMarkdown": "Their main difference is that early stopping is \"stop if learning isn't happening\" whereas saving is \"given training cycle get best validation-scoring model\". It is possible that they may give the same model, but since they both do require some external tuning (what to assess on? How many epochs to wait till it actually stops? (this is referred as patience usually), etc so they can certainly provide different results. What I like to do is to do saving based on validation at all times to get the best valid scoring ckpt, while applying early stopping on top of it if I am time-restricted, or cases where it could be that the model converged before even 1/3 of epochs specified has reached (hence to prevent wasting time). Hope that gives you a better idea about the two :D", "votes": null }, { "id": "748686", "postDate": "02/17/2020 20:51:30", "content": "@joonl04 Thank you very much!
", "rawMarkdown": "joonl04 Thank you very much!", "votes": null } ], "comments": [ { "id": 745621, "author_name": "inoueu1", "author_url": "", "post_date": "02/14/2020 02:19:05", "content": "To my understanding, they are different. Early stopping gets a model, some steps after the monitoring score doesn't improve. So the model is not a best validation scoring model. And, if you save models based on validation, it is not always the last epoch model. Of course, you can combine these two.
\n\nIn my case of this competition, I don't use early stopping, but save models based on validation.\nThis is because... when I got unexpectedly high score in the early epochs, if using early stopping, the model stop training soon after the high validation score even when it is not fully trained by train dataset.
\n\nPlease correct if I am wrong🙏
", "votes": null, "replies": [ { "id": 745632, "author_name": "tonychenxyz", "author_url": "", "post_date": "02/14/2020 02:50:03", "content": "Yeah that's what I was thinking too. Thanks! @inoueu1 I am asking this because I found that my codes for this competition don't seem to work perfectly by saving on validation, so I will probably try early stopping.
", "votes": null, "replies": [] } ] }, { "id": 748633, "author_name": "joonl04", "author_url": "", "post_date": "02/17/2020 19:11:37", "content": "Their main difference is that early stopping is \"stop if learning isn't happening\" whereas saving is \"given training cycle get best validation-scoring model\". It is possible that they may give the same model, but since they both do require some external tuning (what to assess on? How many epochs to wait till it actually stops? (this is referred as patience usually), etc so they can certainly provide different results. What I like to do is to do saving based on validation at all times to get the best valid scoring ckpt, while applying early stopping on top of it if I am time-restricted, or cases where it could be that the model converged before even 1/3 of epochs specified has reached (hence to prevent wasting time). Hope that gives you a better idea about the two :D
", "votes": null, "replies": [ { "id": 748686, "author_name": "tonychenxyz", "author_url": "", "post_date": "02/17/2020 20:51:30", "content": "@joonl04 Thank you very much!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "745493": "They should be the same, right? I got confused.", "745621": "To my understanding, they are different. Early stopping gets a model, some steps after the monitoring score doesn't improve. So the model is not a best validation scoring model. And, if you save models based on validation, it is not always the last epoch model. Of course, you can combine these two.\n\nIn my case of this competition, I don't use early stopping, but save models based on validation.\nThis is because... when I got unexpectedly high score in the early epochs, if using early stopping, the model stop training soon after the high validation score even when it is not fully trained by train dataset.\n\nPlease correct if I am wrong🙏", "745632": "Yeah that's what I was thinking too. Thanks! @inoueu1 I am asking this because I found that my codes for this competition don't seem to work perfectly by saving on validation, so I will probably try early stopping.", "748633": "Their main difference is that early stopping is \"stop if learning isn't happening\" whereas saving is \"given training cycle get best validation-scoring model\". It is possible that they may give the same model, but since they both do require some external tuning (what to assess on? How many epochs to wait till it actually stops? (this is referred as patience usually), etc so they can certainly provide different results. What I like to do is to do saving based on validation at all times to get the best valid scoring ckpt, while applying early stopping on top of it if I am time-restricted, or cases where it could be that the model converged before even 1/3 of epochs specified has reached (hence to prevent wasting time). Hope that gives you a better idea about the two :D", "748686": "joonl04 Thank you very much!" }, "source": "meta" }