What is the maximum batch_size did you use for your model?
", "messages": [ { "id": "732176", "postDate": "01/29/2020 14:46:12", "content": "What is the maximum batch_size did you use for your model?
", "rawMarkdown": "What is the maximum batch_size did you use for your model?", "votes": null }, { "id": "732180", "postDate": "01/29/2020 14:47:50", "content": "Mine : \nModel = Resnet 101 (single model)\nInput size = 128 * 128\nbatch_size = 130
", "rawMarkdown": "Mine : \nModel = Resnet 101 (single model)\nInput size = 128 * 128\nbatch_size = 130", "votes": null }, { "id": "732412", "postDate": "01/29/2020 19:38:42", "content": "On ResNet50 and DenseNet121 I manage to go all the way up to 512 on 75x75 images. Haven't tried multiple batch size / image sizes yet to see the effect; so far I'm mostly going for the quickest training times possible.
", "rawMarkdown": "On ResNet50 and DenseNet121 I manage to go all the way up to 512 on 75x75 images. Haven't tried multiple batch size / image sizes yet to see the effect; so far I'm mostly going for the quickest training times possible.", "votes": null }, { "id": "735835", "postDate": "02/03/2020 13:59:22", "content": "@maxlenormand That is very impressive! I cannot seem to go higher than 32 batches at 128x128 size using Colab's resources and DenseNet201 ... What do you think abou the use of 75x75 sized images instead to save some memory ?
", "rawMarkdown": "maxlenormand That is very impressive! I cannot seem to go higher than 32 batches at 128x128 size using Colab's resources and DenseNet201 ... What do you think abou the use of 75x75 sized images instead to save some memory ?", "votes": null }, { "id": "735929", "postDate": "02/03/2020 16:07:27", "content": "Oh this was on Kaggle kernels, so I don´t know if the GPUs have as much memory on both platforms? I know that the GPU you get allocated on Colab can change from on session to another.
\n\nAt the moment I'm trying to increase my batch size and reduce my image sizes to either increase my training time (higher number of epochs before hitting the kernel limit. Increasing epochs has allowed me to gain a few thousands of a point) or simply make training faster with a higher batch size.
\n\nI'm never training on 128x128. I've done either 100x100 or 75x75. For now 75x75 seems pretty nice as it's about half the size in terms of pixels than a 100x100 image and I see very little difference. You also do save about half the amount of memory. With the limited hardware available it seems like a good sweet spot that I found.
\n\nThere's probably some gains to be had on the pre-processing of images though. We seem to be both stuck at the same score @dimartinot so I'm guessing we are hitting the same wall and having a hard time improving our scores?
", "rawMarkdown": "Oh this was on Kaggle kernels, so I don´t know if the GPUs have as much memory on both platforms? I know that the GPU you get allocated on Colab can change from on session to another.\n\nAt the moment I'm trying to increase my batch size and reduce my image sizes to either increase my training time (higher number of epochs before hitting the kernel limit. Increasing epochs has allowed me to gain a few thousands of a point) or simply make training faster with a higher batch size.\n\nI'm never training on 128x128. I've done either 100x100 or 75x75. For now 75x75 seems pretty nice as it's about half the size in terms of pixels than a 100x100 image and I see very little difference. You also do save about half the amount of memory. With the limited hardware available it seems like a good sweet spot that I found.\n\nThere's probably some gains to be had on the pre-processing of images though. We seem to be both stuck at the same score @dimartinot so I'm guessing we are hitting the same wall and having a hard time improving our scores?", "votes": null } ], "comments": [ { "id": 732180, "author_name": "karan07", "author_url": "", "post_date": "01/29/2020 14:47:50", "content": "Mine : \nModel = Resnet 101 (single model)\nInput size = 128 * 128\nbatch_size = 130
", "votes": null, "replies": [] }, { "id": 732412, "author_name": "maxlenormand", "author_url": "", "post_date": "01/29/2020 19:38:42", "content": "On ResNet50 and DenseNet121 I manage to go all the way up to 512 on 75x75 images. Haven't tried multiple batch size / image sizes yet to see the effect; so far I'm mostly going for the quickest training times possible.
", "votes": null, "replies": [ { "id": 735835, "author_name": "dimartinot", "author_url": "", "post_date": "02/03/2020 13:59:22", "content": "@maxlenormand That is very impressive! I cannot seem to go higher than 32 batches at 128x128 size using Colab's resources and DenseNet201 ... What do you think abou the use of 75x75 sized images instead to save some memory ?
", "votes": null, "replies": [] }, { "id": 735929, "author_name": "maxlenormand", "author_url": "", "post_date": "02/03/2020 16:07:27", "content": "Oh this was on Kaggle kernels, so I don´t know if the GPUs have as much memory on both platforms? I know that the GPU you get allocated on Colab can change from on session to another.
\n\nAt the moment I'm trying to increase my batch size and reduce my image sizes to either increase my training time (higher number of epochs before hitting the kernel limit. Increasing epochs has allowed me to gain a few thousands of a point) or simply make training faster with a higher batch size.
\n\nI'm never training on 128x128. I've done either 100x100 or 75x75. For now 75x75 seems pretty nice as it's about half the size in terms of pixels than a 100x100 image and I see very little difference. You also do save about half the amount of memory. With the limited hardware available it seems like a good sweet spot that I found.
\n\nThere's probably some gains to be had on the pre-processing of images though. We seem to be both stuck at the same score @dimartinot so I'm guessing we are hitting the same wall and having a hard time improving our scores?
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "732176": "What is the maximum batch_size did you use for your model?", "732180": "Mine : \nModel = Resnet 101 (single model)\nInput size = 128 * 128\nbatch_size = 130", "732412": "On ResNet50 and DenseNet121 I manage to go all the way up to 512 on 75x75 images. Haven't tried multiple batch size / image sizes yet to see the effect; so far I'm mostly going for the quickest training times possible.", "735835": "maxlenormand That is very impressive! I cannot seem to go higher than 32 batches at 128x128 size using Colab's resources and DenseNet201 ... What do you think abou the use of 75x75 sized images instead to save some memory ?", "735929": "Oh this was on Kaggle kernels, so I don´t know if the GPUs have as much memory on both platforms? I know that the GPU you get allocated on Colab can change from on session to another.\n\nAt the moment I'm trying to increase my batch size and reduce my image sizes to either increase my training time (higher number of epochs before hitting the kernel limit. Increasing epochs has allowed me to gain a few thousands of a point) or simply make training faster with a higher batch size.\n\nI'm never training on 128x128. I've done either 100x100 or 75x75. For now 75x75 seems pretty nice as it's about half the size in terms of pixels than a 100x100 image and I see very little difference. You also do save about half the amount of memory. With the limited hardware available it seems like a good sweet spot that I found.\n\nThere's probably some gains to be had on the pre-processing of images though. We seem to be both stuck at the same score @dimartinot so I'm guessing we are hitting the same wall and having a hard time improving our scores?" }, "source": "meta" }