After training for 15 epochs, it took about 9 hours. The training accuracy is 99%. Is it normal?
", "messages": [ { "id": "1148281", "postDate": "01/11/2021 03:45:42", "content": "After training for 15 epochs, it took about 9 hours. The training accuracy is 99%. Is it normal?
", "rawMarkdown": "After training for 15 epochs, it took about 9 hours. The training accuracy is 99%. Is it normal?", "votes": null }, { "id": "1148522", "postDate": "01/11/2021 07:33:42", "content": "This is a text book example of overfitting. Try to check if your validation is also in the same range. If it isn't try to check your learning rate and dropout rate. Increase dropout rate for regularization and generalization to validation data. High Learning rate could lead to overfitting very fast.
", "rawMarkdown": "This is a text book example of overfitting. Try to check if your validation is also in the same range. If it isn't try to check your learning rate and dropout rate. Increase dropout rate for regularization and generalization to validation data. High Learning rate could lead to overfitting very fast.", "votes": null }, { "id": "1148526", "postDate": "01/11/2021 07:36:59", "content": "Sure, why not? If you are only overfitting enough, you can get your training accuracy close to (or exactly to) 100% relatively fast. However, what you should mostly care about is validation accuracy (i.e. in a reserved validation set, either a single validation set or better a cross-validation).
\nI suspect such a high training accuracy is likely a bad thing, because your model is more likely to be learning to \"memorize\" the correct answer to each training image than learning features that generalize well. I'm mostly saying that, because when you look at the leaderboard, you do not see any performance that good, so unless you've come up with something amazing everyone has missed, your training accuracy is more likely to be overfitting than truly fantastic performance and I would expect that you will see much worse performance on a validation set.
\n9 hours training is plausible depending on what model, what augementation, what hardware and how efficient a set-up you are using. Many smaller models should train in <= 1 to 2 hours with one GPU and for your first experimentations you may not want to go to the biggest model right away.
", "rawMarkdown": "Sure, why not? If you are only overfitting enough, you can get your training accuracy close to (or exactly to) 100% relatively fast. However, what you should mostly care about is validation accuracy (i.e. in a reserved validation set, either a single validation set or better a cross-validation).\n\nI suspect such a high training accuracy is likely a bad thing, because your model is more likely to be learning to \"memorize\" the correct answer to each training image than learning features that generalize well. I'm mostly saying that, because when you look at the leaderboard, you do not see any performance that good, so unless you've come up with something amazing everyone has missed, your training accuracy is more likely to be overfitting than truly fantastic performance and I would expect that you will see much worse performance on a validation set.\n\n9 hours training is plausible depending on what model, what augementation, what hardware and how efficient a set-up you are using. Many smaller models should train in <= 1 to 2 hours with one GPU and for your first experimentations you may not want to go to the biggest model right away.", "votes": null }, { "id": "1148560", "postDate": "01/11/2021 08:12:12", "content": "What is your validation accuracy @henini ?
", "rawMarkdown": "What is your validation accuracy @henini ?", "votes": null }, { "id": "1148942", "postDate": "01/11/2021 13:44:46", "content": "With such noisy label, +99% training accuracy is not a good thing. This means not only the model is overfitting by memorizing labels, but also worstly, it is memorizing the noisy labels.
", "rawMarkdown": "With such noisy label, +99% training accuracy is not a good thing. This means not only the model is overfitting by memorizing labels, but also worstly, it is memorizing the noisy labels.", "votes": null }, { "id": "1149657", "postDate": "01/12/2021 03:39:02", "content": "You are definitely overfitting, try reading up on ways for your model to prevent overfitting. You should also always make sure that you're using a validation set, otherwise you won't know how good your model is performing. Training accuracy usually doesn't mean much, you will probably do better in this competition by having it lower, but performing just as good as the validation set.
", "rawMarkdown": "You are definitely overfitting, try reading up on ways for your model to prevent overfitting. You should also always make sure that you're using a validation set, otherwise you won't know how good your model is performing. Training accuracy usually doesn't mean much, you will probably do better in this competition by having it lower, but performing just as good as the validation set.", "votes": null }, { "id": "1151008", "postDate": "01/13/2021 04:12:54", "content": "Your model is badly overfitting and even worse it is overfitting to noise. You can do following such as:
\nIn the case of 20% verification set, why can I only achieve 93.5-94% accuracy no matter how I train in this project? Don't you add layers to prevent over fitting or data enhancement? For example, dropout, L2 regularization, BN layer, and some image enhancement
", "rawMarkdown": "In the case of 20% verification set, why can I only achieve 93.5-94% accuracy no matter how I train in this project? Don't you add layers to prevent over fitting or data enhancement? For example, dropout, L2 regularization, BN layer, and some image enhancement", "votes": null } ], "comments": [ { "id": 1148522, "author_name": "manikanthr5", "author_url": "", "post_date": "01/11/2021 07:33:42", "content": "This is a text book example of overfitting. Try to check if your validation is also in the same range. If it isn't try to check your learning rate and dropout rate. Increase dropout rate for regularization and generalization to validation data. High Learning rate could lead to overfitting very fast.
", "votes": null, "replies": [] }, { "id": 1148526, "author_name": "bjoernholzhauer", "author_url": "", "post_date": "01/11/2021 07:36:59", "content": "Sure, why not? If you are only overfitting enough, you can get your training accuracy close to (or exactly to) 100% relatively fast. However, what you should mostly care about is validation accuracy (i.e. in a reserved validation set, either a single validation set or better a cross-validation).
\nI suspect such a high training accuracy is likely a bad thing, because your model is more likely to be learning to \"memorize\" the correct answer to each training image than learning features that generalize well. I'm mostly saying that, because when you look at the leaderboard, you do not see any performance that good, so unless you've come up with something amazing everyone has missed, your training accuracy is more likely to be overfitting than truly fantastic performance and I would expect that you will see much worse performance on a validation set.
\n9 hours training is plausible depending on what model, what augementation, what hardware and how efficient a set-up you are using. Many smaller models should train in <= 1 to 2 hours with one GPU and for your first experimentations you may not want to go to the biggest model right away.
", "votes": null, "replies": [ { "id": 1148942, "author_name": "serigne", "author_url": "", "post_date": "01/11/2021 13:44:46", "content": "With such noisy label, +99% training accuracy is not a good thing. This means not only the model is overfitting by memorizing labels, but also worstly, it is memorizing the noisy labels.
", "votes": null, "replies": [] } ] }, { "id": 1148560, "author_name": "ishalgarg", "author_url": "", "post_date": "01/11/2021 08:12:12", "content": "What is your validation accuracy @henini ?
", "votes": null, "replies": [] }, { "id": 1149657, "author_name": "capiru", "author_url": "", "post_date": "01/12/2021 03:39:02", "content": "You are definitely overfitting, try reading up on ways for your model to prevent overfitting. You should also always make sure that you're using a validation set, otherwise you won't know how good your model is performing. Training accuracy usually doesn't mean much, you will probably do better in this competition by having it lower, but performing just as good as the validation set.
", "votes": null, "replies": [] }, { "id": 1151008, "author_name": "raghaw", "author_url": "", "post_date": "01/13/2021 04:12:54", "content": "Your model is badly overfitting and even worse it is overfitting to noise. You can do following such as:
\nIn the case of 20% verification set, why can I only achieve 93.5-94% accuracy no matter how I train in this project? Don't you add layers to prevent over fitting or data enhancement? For example, dropout, L2 regularization, BN layer, and some image enhancement
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1148281": "After training for 15 epochs, it took about 9 hours. The training accuracy is 99%. Is it normal?", "1148522": "This is a text book example of overfitting. Try to check if your validation is also in the same range. If it isn't try to check your learning rate and dropout rate. Increase dropout rate for regularization and generalization to validation data. High Learning rate could lead to overfitting very fast.", "1148526": "Sure, why not? If you are only overfitting enough, you can get your training accuracy close to (or exactly to) 100% relatively fast. However, what you should mostly care about is validation accuracy (i.e. in a reserved validation set, either a single validation set or better a cross-validation).\n\nI suspect such a high training accuracy is likely a bad thing, because your model is more likely to be learning to \"memorize\" the correct answer to each training image than learning features that generalize well. I'm mostly saying that, because when you look at the leaderboard, you do not see any performance that good, so unless you've come up with something amazing everyone has missed, your training accuracy is more likely to be overfitting than truly fantastic performance and I would expect that you will see much worse performance on a validation set.\n\n9 hours training is plausible depending on what model, what augementation, what hardware and how efficient a set-up you are using. Many smaller models should train in <= 1 to 2 hours with one GPU and for your first experimentations you may not want to go to the biggest model right away.", "1148560": "What is your validation accuracy @henini ?", "1148942": "With such noisy label, +99% training accuracy is not a good thing. This means not only the model is overfitting by memorizing labels, but also worstly, it is memorizing the noisy labels.", "1149657": "You are definitely overfitting, try reading up on ways for your model to prevent overfitting. You should also always make sure that you're using a validation set, otherwise you won't know how good your model is performing. Training accuracy usually doesn't mean much, you will probably do better in this competition by having it lower, but performing just as good as the validation set.", "1151008": "Your model is badly overfitting and even worse it is overfitting to noise. You can do following such as:\n1. Most importantly split the data into training set and validation set and observe the validation sets accuracy.\n2. Image Augmentations.\n3. Smaller models.\n4. Noise robust loss functions.\n5. Early stopping on validation set accuracy.", "1151085": "In the case of 20% verification set, why can I only achieve 93.5-94% accuracy no matter how I train in this project? Don't you add layers to prevent over fitting or data enhancement? For example, dropout, L2 regularization, BN layer, and some image enhancement" }, "source": "meta" }