Since the dataset is huge and one batch of training takes around ~25 minutes(for me) with efficientnet encoders, I think it would be helpful if we share our tips/tricks to experiment/iterate fast(er) of our ideas. Let me start: \n1. Use smaller image size(256, even 128)\n2. Use subset of data \n3. Use lighter encoders(start with eff_b0 instead of eff_b4+)
\n\n", "messages": [ { "id": "875939", "postDate": "06/06/2020 09:50:05", "content": "Since the dataset is huge and one batch of training takes around ~25 minutes(for me) with efficientnet encoders, I think it would be helpful if we share our tips/tricks to experiment/iterate fast(er) of our ideas. Let me start: \n1. Use smaller image size(256, even 128)\n2. Use subset of data \n3. Use lighter encoders(start with eff_b0 instead of eff_b4+)
\n\n", "rawMarkdown": "Since the dataset is huge and one batch of training takes around ~25 minutes(for me) with efficientnet encoders, I think it would be helpful if we share our tips/tricks to experiment/iterate fast(er) of our ideas. Let me start: \n1. Use smaller image size(256, even 128)\n2. Use subset of data \n3. Use lighter encoders(start with eff_b0 instead of eff_b4+)\n\n", "votes": null }, { "id": "876311", "postDate": "06/06/2020 16:06:05", "content": "1 will not work. Resize harms the training.
", "rawMarkdown": "1 will not work. Resize harms the training.", "votes": null }, { "id": "876379", "postDate": "06/06/2020 16:41:17", "content": "rule No1 - read the forum
", "rawMarkdown": "rule No1 - read the forum", "votes": null }, { "id": "876446", "postDate": "06/06/2020 17:32:05", "content": "Mixed Precision
", "rawMarkdown": "Mixed Precision", "votes": null }, { "id": "876780", "postDate": "06/07/2020 03:33:04", "content": "First, you don’t have to resize images it’s mentioned already secondly if one batch is taking 25 min then I think you have not enabled gpu.
", "rawMarkdown": "First, you don’t have to resize images it’s mentioned already secondly if one batch is taking 25 min then I think you have not enabled gpu.", "votes": null }, { "id": "876785", "postDate": "06/07/2020 03:36:19", "content": "If you have enabled GPU/TPU then adding batchnormalization layer can speed up the learning
", "rawMarkdown": "If you have enabled GPU/TPU then adding batchnormalization layer can speed up the learning", "votes": null }, { "id": "878155", "postDate": "06/08/2020 09:50:23", "content": "Converting JPGs into tfrecords
", "rawMarkdown": "Converting JPGs into tfrecords", "votes": null }, { "id": "878817", "postDate": "06/08/2020 21:55:38", "content": "Mixed-precision training and lighter models with bigger batch can help you. Eugene highlighted in this discussion these tips that allow you to speed up the process of training models.
", "rawMarkdown": "Mixed-precision training and lighter models with bigger batch can help you. Eugene highlighted in [this discussion](https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/155392) these tips that allow you to speed up the process of training models.", "votes": null }, { "id": "918077", "postDate": "07/07/2020 00:58:51", "content": "@bibek777 - I am using a subsample of the dataset to see if I can generate quick experiments. Also, I am using apex to use larger batch sizes which is making training a bit quicker. I am pretty much a beginner when it comes to image competitions so these suggestions may or may not work.
1 will not work. Resize harms the training.
", "votes": null, "replies": [] }, { "id": 876379, "author_name": "valanm", "author_url": "", "post_date": "06/06/2020 16:41:17", "content": "rule No1 - read the forum
", "votes": null, "replies": [] }, { "id": 876446, "author_name": "nuller", "author_url": "", "post_date": "06/06/2020 17:32:05", "content": "Mixed Precision
", "votes": null, "replies": [] }, { "id": 876780, "author_name": "priyt00", "author_url": "", "post_date": "06/07/2020 03:33:04", "content": "First, you don’t have to resize images it’s mentioned already secondly if one batch is taking 25 min then I think you have not enabled gpu.
", "votes": null, "replies": [ { "id": 876785, "author_name": "priyt00", "author_url": "", "post_date": "06/07/2020 03:36:19", "content": "If you have enabled GPU/TPU then adding batchnormalization layer can speed up the learning
", "votes": null, "replies": [] } ] }, { "id": 878155, "author_name": "zhenglv", "author_url": "", "post_date": "06/08/2020 09:50:23", "content": "Converting JPGs into tfrecords
", "votes": null, "replies": [] }, { "id": 878817, "author_name": "gofixyourself", "author_url": "", "post_date": "06/08/2020 21:55:38", "content": "Mixed-precision training and lighter models with bigger batch can help you. Eugene highlighted in this discussion these tips that allow you to speed up the process of training models.
", "votes": null, "replies": [] }, { "id": 918077, "author_name": "rdizzl3", "author_url": "", "post_date": "07/07/2020 00:58:51", "content": "@bibek777 - I am using a subsample of the dataset to see if I can generate quick experiments. Also, I am using apex to use larger batch sizes which is making training a bit quicker. I am pretty much a beginner when it comes to image competitions so these suggestions may or may not work.