Hi, I was trying to ensemble 5-Fold models, for small resolution images I was successful, but when I tried with higher resolution images I'm getting a memory error here. I got an error with the following approaches:
\n\nCreating a single model that was an ensemble of the 5 others (this was the best option because I ended with a single model)
Iterating on a list of models and making predictions.
```\npreds = np.zeros((len(df), HEIGHT, WIDTH, N_CLASSES))\nfor segmentation_model in segmentation_model_list:\n preds += segmentation_model.predict_generator(inspect_generator)
\n\npreds /= len(segmentation_model_list)\n```
\n\nDoes anyone have a suggestion about how to do this?
", "messages": [ { "id": "653774", "postDate": "10/21/2019 00:03:18", "content": "Hi, I was trying to ensemble 5-Fold models, for small resolution images I was successful, but when I tried with higher resolution images I'm getting a memory error here. I got an error with the following approaches:
\n\nCreating a single model that was an ensemble of the 5 others (this was the best option because I ended with a single model)
Iterating on a list of models and making predictions.
```\npreds = np.zeros((len(df), HEIGHT, WIDTH, N_CLASSES))\nfor segmentation_model in segmentation_model_list:\n preds += segmentation_model.predict_generator(inspect_generator)
\n\npreds /= len(segmentation_model_list)\n```
\n\nDoes anyone have a suggestion about how to do this?
", "rawMarkdown": "Hi, I was trying to ensemble 5-Fold models, for small resolution images I was successful, but when I tried with higher resolution images I'm getting a memory error here. I got an error with the following approaches:\n\n1. Creating a single model that was an ensemble of the 5 others (this was the best option because I ended with a single model)\n\n2. Iterating on a list of models and making predictions.\n\n```\npreds = np.zeros((len(df), HEIGHT, WIDTH, N_CLASSES))\nfor segmentation_model in segmentation_model_list:\n preds += segmentation_model.predict_generator(inspect_generator)\n\npreds /= len(segmentation_model_list)\n```\n\nDoes anyone have a suggestion about how to do this?", "votes": null }, { "id": "654503", "postDate": "10/22/2019 00:15:36", "content": "Weighted average and simple average, I prefer weighted average
", "rawMarkdown": "Weighted average and simple average, I prefer weighted average", "votes": null }, { "id": "654512", "postDate": "10/22/2019 00:50:46", "content": "Hi @zhangeng , I don't mean the technique, but how you handle loading all models and making predictions, when I try to do it I get memory error and my kernel dies.
", "rawMarkdown": "Hi @zhangeng , I don't mean the technique, but how you handle loading all models and making predictions, when I try to do it I get memory error and my kernel dies.", "votes": null }, { "id": "654911", "postDate": "10/22/2019 13:27:27", "content": "You have to do it with batches... for example, you take 500 samples and predict for each model, merge, clean memory and repeat for another 500
", "rawMarkdown": "You have to do it with batches... for example, you take 500 samples and predict for each model, merge, clean memory and repeat for another 500", "votes": null }, { "id": "655004", "postDate": "10/22/2019 15:25:57", "content": "Hi Igor, thanks for the response, but I'm already doing this, in fact, I'm using batches of size 300, in my case just by loading the 5 models, my RAM is almost all full, but I'll try even smaller batches and maybe using garbage collector may help
", "rawMarkdown": "Hi Igor, thanks for the response, but I'm already doing this, in fact, I'm using batches of size 300, in my case just by loading the 5 models, my RAM is almost all full, but I'll try even smaller batches and maybe using garbage collector may help", "votes": null }, { "id": "655162", "postDate": "10/22/2019 18:48:09", "content": "Write a python script to load a single model and write prediction in a csv file.\nRun this script for any number of models (only one model is in the memory at a time)\nFinally, make a script to average all these csvs. \nIt would be much easier to not write separate scripts and do all this in a single notebook but for some reasons even if I can clear the RAM but I still end up getting CUDA out of memory (I am no expert). \nWriting scripts and running from terminal is the only fix I have.\nWrite a bash to run all these scripts sequentially, will save you some trouble.
", "rawMarkdown": "Write a python script to load a single model and write prediction in a csv file.\nRun this script for any number of models (only one model is in the memory at a time)\nFinally, make a script to average all these csvs. \nIt would be much easier to not write separate scripts and do all this in a single notebook but for some reasons even if I can clear the RAM but I still end up getting CUDA out of memory (I am no expert). \nWriting scripts and running from terminal is the only fix I have.\nWrite a bash to run all these scripts sequentially, will save you some trouble.", "votes": null }, { "id": "655197", "postDate": "10/22/2019 19:43:12", "content": "Thanks @timetraveller98 , I was thinking to do something like this, but I was trying to avoid these extreme measures, but maybe making batch predictions and saving them to a file, and then loading and averaging may be the option.
", "rawMarkdown": "Thanks @timetraveller98 , I was thinking to do something like this, but I was trying to avoid these extreme measures, but maybe making batch predictions and saving them to a file, and then loading and averaging may be the option.", "votes": null }, { "id": "655395", "postDate": "10/23/2019 02:25:04", "content": "Try to convert the value to float16 for each prediction.
", "rawMarkdown": "Try to convert the value to float16 for each prediction.", "votes": null }, { "id": "655454", "postDate": "10/23/2019 04:21:45", "content": "Or we can multiply prediction by 100, and convert it to np.int8 or uint8. That will save more than 80% capacity.
", "rawMarkdown": "Or we can multiply prediction by 100, and convert it to np.int8 or uint8. That will save more than 80% capacity.", "votes": null }, { "id": "655746", "postDate": "10/23/2019 12:57:44", "content": "Thanks @d46kobayashi and @khahuras both are very valid solutions
", "rawMarkdown": "Thanks @d46kobayashi and @khahuras both are very valid solutions", "votes": null }, { "id": "660773", "postDate": "10/29/2019 15:23:13", "content": "😀 Create a Gmail and use the free credit to rent a VM on GCP with large memory...Or you can create Jupyter Notebook on GCP as well.
\n\n", "rawMarkdown": "😀 Create a Gmail and use the free credit to rent a VM on GCP with large memory...Or you can create Jupyter Notebook on GCP as well.\n\nhttps://cloud.google.com/ai-platform/notebooks/docs/?_ga=2.130818060.-2067446994.1542988993&_gac=1.224661096.1572362686.CjwKCAjwxt_tBRAXEiwAENY8heyti_nNSfk2jbSIlEiYin0HjmAkcPPBeZ0HYWYsmLiB2TSrjwNAjhoCfxMQAvD_BwE", "votes": null }, { "id": "660883", "postDate": "10/29/2019 18:26:54", "content": "The notebook here shows an example. The outer for-loop uses batch_size=2. Then inside each iteration, predict all your Keras models. Then average the masks and convert to a single rle and add that rle to a python list. When your for-loop ends, convert your entire rle python list into your final dataframe submission.
(This example is in PyTorch but you can convert the idea to Keras).
", "rawMarkdown": "The notebook [here][1] shows an example. The outer for-loop uses batch_size=2. Then inside each iteration, predict all your Keras models. Then average the masks and convert to a single `rle` and add that `rle` to a python list. When your for-loop ends, convert your entire `rle` python list into your final dataframe submission.\n\n(This example is in PyTorch but you can convert the idea to Keras).\n\n[1]: https://www.kaggle.com/lightforever/severstal-mlcomp-catalyst-infer-0-90672", "votes": null }, { "id": "660907", "postDate": "10/29/2019 19:05:46", "content": "Do the probability average/ensemble within batch.
", "rawMarkdown": "Do the probability average/ensemble within batch.", "votes": null }, { "id": "661155", "postDate": "10/30/2019 00:50:23", "content": "Thanks @cdeotte and @naivelamb , that's basically what I'm doing, I managed to make de predictions on Colab and using batch size of 100
", "rawMarkdown": "Thanks @cdeotte and @naivelamb , that's basically what I'm doing, I managed to make de predictions on Colab and using batch size of 100", "votes": null } ], "comments": [ { "id": 654503, "author_name": "zhangeng", "author_url": "", "post_date": "10/22/2019 00:15:36", "content": "Weighted average and simple average, I prefer weighted average
", "votes": null, "replies": [ { "id": 654512, "author_name": "dimitreoliveira", "author_url": "", "post_date": "10/22/2019 00:50:46", "content": "Hi @zhangeng , I don't mean the technique, but how you handle loading all models and making predictions, when I try to do it I get memory error and my kernel dies.
", "votes": null, "replies": [] } ] }, { "id": 654911, "author_name": "igormunizims", "author_url": "", "post_date": "10/22/2019 13:27:27", "content": "You have to do it with batches... for example, you take 500 samples and predict for each model, merge, clean memory and repeat for another 500
", "votes": null, "replies": [ { "id": 655004, "author_name": "dimitreoliveira", "author_url": "", "post_date": "10/22/2019 15:25:57", "content": "Hi Igor, thanks for the response, but I'm already doing this, in fact, I'm using batches of size 300, in my case just by loading the 5 models, my RAM is almost all full, but I'll try even smaller batches and maybe using garbage collector may help
", "votes": null, "replies": [] } ] }, { "id": 655162, "author_name": "timetraveller98", "author_url": "", "post_date": "10/22/2019 18:48:09", "content": "Write a python script to load a single model and write prediction in a csv file.\nRun this script for any number of models (only one model is in the memory at a time)\nFinally, make a script to average all these csvs. \nIt would be much easier to not write separate scripts and do all this in a single notebook but for some reasons even if I can clear the RAM but I still end up getting CUDA out of memory (I am no expert). \nWriting scripts and running from terminal is the only fix I have.\nWrite a bash to run all these scripts sequentially, will save you some trouble.
", "votes": null, "replies": [ { "id": 655197, "author_name": "dimitreoliveira", "author_url": "", "post_date": "10/22/2019 19:43:12", "content": "Thanks @timetraveller98 , I was thinking to do something like this, but I was trying to avoid these extreme measures, but maybe making batch predictions and saving them to a file, and then loading and averaging may be the option.
", "votes": null, "replies": [] } ] }, { "id": 655395, "author_name": "d46kobayashi", "author_url": "", "post_date": "10/23/2019 02:25:04", "content": "Try to convert the value to float16 for each prediction.
", "votes": null, "replies": [ { "id": 655454, "author_name": "khahuras", "author_url": "", "post_date": "10/23/2019 04:21:45", "content": "Or we can multiply prediction by 100, and convert it to np.int8 or uint8. That will save more than 80% capacity.
", "votes": null, "replies": [] }, { "id": 655746, "author_name": "dimitreoliveira", "author_url": "", "post_date": "10/23/2019 12:57:44", "content": "Thanks @d46kobayashi and @khahuras both are very valid solutions
", "votes": null, "replies": [] } ] }, { "id": 660773, "author_name": "gogo827jz", "author_url": "", "post_date": "10/29/2019 15:23:13", "content": "😀 Create a Gmail and use the free credit to rent a VM on GCP with large memory...Or you can create Jupyter Notebook on GCP as well.
\n\n", "votes": null, "replies": [] }, { "id": 660883, "author_name": "cdeotte", "author_url": "", "post_date": "10/29/2019 18:26:54", "content": "The notebook here shows an example. The outer for-loop uses batch_size=2. Then inside each iteration, predict all your Keras models. Then average the masks and convert to a single rle and add that rle to a python list. When your for-loop ends, convert your entire rle python list into your final dataframe submission.
(This example is in PyTorch but you can convert the idea to Keras).
", "votes": null, "replies": [ { "id": 661155, "author_name": "dimitreoliveira", "author_url": "", "post_date": "10/30/2019 00:50:23", "content": "Thanks @cdeotte and @naivelamb , that's basically what I'm doing, I managed to make de predictions on Colab and using batch size of 100
", "votes": null, "replies": [] } ] }, { "id": 660907, "author_name": "naivelamb", "author_url": "", "post_date": "10/29/2019 19:05:46", "content": "Do the probability average/ensemble within batch.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "653774": "Hi, I was trying to ensemble 5-Fold models, for small resolution images I was successful, but when I tried with higher resolution images I'm getting a memory error here. I got an error with the following approaches:\n\n1. Creating a single model that was an ensemble of the 5 others (this was the best option because I ended with a single model)\n\n2. Iterating on a list of models and making predictions.\n\n```\npreds = np.zeros((len(df), HEIGHT, WIDTH, N_CLASSES))\nfor segmentation_model in segmentation_model_list:\n preds += segmentation_model.predict_generator(inspect_generator)\n\npreds /= len(segmentation_model_list)\n```\n\nDoes anyone have a suggestion about how to do this?", "654503": "Weighted average and simple average, I prefer weighted average", "654512": "Hi @zhangeng , I don't mean the technique, but how you handle loading all models and making predictions, when I try to do it I get memory error and my kernel dies.", "654911": "You have to do it with batches... for example, you take 500 samples and predict for each model, merge, clean memory and repeat for another 500", "655004": "Hi Igor, thanks for the response, but I'm already doing this, in fact, I'm using batches of size 300, in my case just by loading the 5 models, my RAM is almost all full, but I'll try even smaller batches and maybe using garbage collector may help", "655162": "Write a python script to load a single model and write prediction in a csv file.\nRun this script for any number of models (only one model is in the memory at a time)\nFinally, make a script to average all these csvs. \nIt would be much easier to not write separate scripts and do all this in a single notebook but for some reasons even if I can clear the RAM but I still end up getting CUDA out of memory (I am no expert). \nWriting scripts and running from terminal is the only fix I have.\nWrite a bash to run all these scripts sequentially, will save you some trouble.", "655197": "Thanks @timetraveller98 , I was thinking to do something like this, but I was trying to avoid these extreme measures, but maybe making batch predictions and saving them to a file, and then loading and averaging may be the option.", "655395": "Try to convert the value to float16 for each prediction.", "655454": "Or we can multiply prediction by 100, and convert it to np.int8 or uint8. That will save more than 80% capacity.", "655746": "Thanks @d46kobayashi and @khahuras both are very valid solutions", "660773": "😀 Create a Gmail and use the free credit to rent a VM on GCP with large memory...Or you can create Jupyter Notebook on GCP as well.\n\nhttps://cloud.google.com/ai-platform/notebooks/docs/?_ga=2.130818060.-2067446994.1542988993&_gac=1.224661096.1572362686.CjwKCAjwxt_tBRAXEiwAENY8heyti_nNSfk2jbSIlEiYin0HjmAkcPPBeZ0HYWYsmLiB2TSrjwNAjhoCfxMQAvD_BwE", "660883": "The notebook [here][1] shows an example. The outer for-loop uses batch_size=2. Then inside each iteration, predict all your Keras models. Then average the masks and convert to a single `rle` and add that `rle` to a python list. When your for-loop ends, convert your entire `rle` python list into your final dataframe submission.\n\n(This example is in PyTorch but you can convert the idea to Keras).\n\n[1]: https://www.kaggle.com/lightforever/severstal-mlcomp-catalyst-infer-0-90672", "660907": "Do the probability average/ensemble within batch.", "661155": "Thanks @cdeotte and @naivelamb , that's basically what I'm doing, I managed to make de predictions on Colab and using batch size of 100" }, "source": "meta" }