I have build CNN model with keras and I am using prediction_generator to predict on test data but prediction time is too long. It almost took 7 hrs to finish whole test set. I am doing resize and normalization of image before doing prediction and using 256*256 size as input. how we can improve on prediction time?
", "messages": [ { "id": 959496, "postDate": "2020-08-05T16:16:01.393Z", "content": "Are you running on Kaggle Notebooks or on your local machine? Also are you using GPUs or TPUs?
\n\nIf you are not using TPUs I really recommend you to try them, they accelerate the training a lot!
", "rawMarkdown": "Are you running on Kaggle Notebooks or on your local machine? Also are you using GPUs or TPUs?\n\nIf you are not using TPUs I really recommend you to try them, they accelerate the training a lot!", "replies": [ { "id": 959502, "postDate": "2020-08-05T16:19:23.080Z", "content": "I am using Kaggle Notebooks and I tried with GPU. Training time for 1 epoch is around 20 mins but prediction time is too long.
", "rawMarkdown": "I am using Kaggle Notebooks and I tried with GPU. Training time for 1 epoch is around 20 mins but prediction time is too long.", "replies": [ { "id": 959554, "postDate": "2020-08-05T17:29:15.527Z", "content": "I think 256 is to hard for GPU, try TPU.\nI can calculate models on GPU for 192*192 or better 128*128, here time not too long. But on 256*256 it becomes painly long
", "rawMarkdown": "I think 256 is to hard for GPU, try TPU.\nI can calculate models on GPU for 192*192 or better 128*128, here time not too long. But on 256*256 it becomes painly long" } ] }, { "id": 959790, "postDate": "2020-08-05T22:26:07.220Z", "content": "Can you post your prediction code? prediction code is usually pretty simple. You simply iterate your batches and pass each to the model and get the prediction out. Perhaps store them in some data structure, maybe do an activation function.......it should go much faster than training as. you don't need to do backwards propogation, batch normalization, etc.
", "rawMarkdown": "Can you post your prediction code? prediction code is usually pretty simple. You simply iterate your batches and pass each to the model and get the prediction out. Perhaps store them in some data structure, maybe do an activation function.......it should go much faster than training as. you don't need to do backwards propogation, batch normalization, etc." }, { "id": 961707, "postDate": "2020-08-07T12:38:50.727Z", "content": "I have written my own generator.\ndef testgenerator(sourcepath,folderlist,batchsize):\nprint(\"Source path is\",sourcepath,\"batch size is\",batchsize)\nwhile True:\nt = np.random.permutation(folderlist) numbatches = len(t)//batchsize for batch in range(numbatches):\nbatchdata = np.zeros((batchsize,y,z,1))\nfor folder in range(batchsize): imgs = sourcepath+'/'+ t[folder + (batchbatchsize)][0]+'.jpg' image = io.imread(imgs).astype(np.float32) image = rgb2gray(image) image = resizeimage(image)\nimage = image.reshape(y,z,1)\nbatchdata[folder,:,:] = image yield batchdata\nbatchdata = np.zeros(((len(t)%batchsize,y,z,1))) #(y,z) is the final size of the input images\nfor folder in range(len(t)%batchsize): imgs = sourcepath+'/'+ t[folder + (batchbatchsize)][0]+'.jpg' image = io.imread(imgs).astype(np.float32) image = rgb2gray(image) image = resizeimage(image)\nimage = image.reshape(y,z,1)\nbatch_data[folder,:,:] = image
\n\nnumtestsequences = len(test)\nprint('# test sequences =', numtestsequences)
\n\nif (numtestsequences%batchsize) == 0: teststeps = int(numtestsequences/batchsize) else: teststeps = (numtestsequences//batch_size) + 1
\n\npredictions = model.predictgenerator(testgen,steps=test_steps)
\n\nWhile training even if it involves back propogation , it completed in around 25 min. prediction should not take much time. Not sure If I am doing anything wrong.
", "rawMarkdown": "I have written my own generator.\ndef testgenerator(sourcepath,folderlist,batchsize):\nprint(\"Source path is\",sourcepath,\"batch size is\",batchsize)\nwhile True:\nt = np.random.permutation(folderlist) numbatches = len(t)//batchsize for batch in range(numbatches):\nbatchdata = np.zeros((batchsize,y,z,1))\nfor folder in range(batchsize): imgs = sourcepath+'/'+ t[folder + (batchbatchsize)][0]+'.jpg' image = io.imread(imgs).astype(np.float32) image = rgb2gray(image) image = resizeimage(image)\nimage = image.reshape(y,z,1)\nbatchdata[folder,:,:] = image yield batchdata\nbatchdata = np.zeros(((len(t)%batchsize,y,z,1))) #(y,z) is the final size of the input images\nfor folder in range(len(t)%batchsize): imgs = sourcepath+'/'+ t[folder + (batchbatchsize)][0]+'.jpg' image = io.imread(imgs).astype(np.float32) image = rgb2gray(image) image = resizeimage(image)\nimage = image.reshape(y,z,1)\nbatch_data[folder,:,:] = image\n\nnumtestsequences = len(test)\nprint('# test sequences =', numtestsequences)\n\nif (numtestsequences%batchsize) == 0: teststeps = int(numtestsequences/batchsize) else: teststeps = (numtestsequences//batch_size) + 1\n\npredictions = model.predictgenerator(testgen,steps=test_steps)\n\nWhile training even if it involves back propogation , it completed in around 25 min. prediction should not take much time. Not sure If I am doing anything wrong." } ] }, { "id": 959481, "postDate": "2020-08-05T16:05:48.030Z", "content": "I have build CNN model with keras and I am using prediction_generator to predict on test data but prediction time is too long. It almost took 7 hrs to finish whole test set. I am doing resize and normalization of image before doing prediction and using 256*256 size as input. how we can improve on prediction time?
", "rawMarkdown": "I have build CNN model with keras and I am using prediction_generator to predict on test data but prediction time is too long. It almost took 7 hrs to finish whole test set. I am doing resize and normalization of image before doing prediction and using 256*256 size as input. how we can improve on prediction time?" }, { "id": 961705, "postDate": "2020-08-07T12:38:15.560Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 959496, "author_name": "Santiago Viquez", "author_url": "", "post_date": "2020-08-05T16:16:01.393000", "content": "Are you running on Kaggle Notebooks or on your local machine? Also are you using GPUs or TPUs?
\n\nIf you are not using TPUs I really recommend you to try them, they accelerate the training a lot!
", "votes": 0, "replies": [ { "id": 959502, "author_name": "Desai Mayank", "author_url": "", "post_date": "2020-08-05T16:19:23.080000", "content": "I am using Kaggle Notebooks and I tried with GPU. Training time for 1 epoch is around 20 mins but prediction time is too long.
", "votes": 0, "replies": [ { "id": 959554, "author_name": "Sxwat", "author_url": "", "post_date": "2020-08-05T17:29:15.527000", "content": "I think 256 is to hard for GPU, try TPU.\nI can calculate models on GPU for 192*192 or better 128*128, here time not too long. But on 256*256 it becomes painly long
", "votes": 0, "replies": [] } ] }, { "id": 959790, "author_name": "Signal", "author_url": "", "post_date": "2020-08-05T22:26:07.220000", "content": "Can you post your prediction code? prediction code is usually pretty simple. You simply iterate your batches and pass each to the model and get the prediction out. Perhaps store them in some data structure, maybe do an activation function.......it should go much faster than training as. you don't need to do backwards propogation, batch normalization, etc.
", "votes": 0, "replies": [] }, { "id": 961707, "author_name": "Desai Mayank", "author_url": "", "post_date": "2020-08-07T12:38:50.727000", "content": "I have written my own generator.\ndef testgenerator(sourcepath,folderlist,batchsize):\nprint(\"Source path is\",sourcepath,\"batch size is\",batchsize)\nwhile True:\nt = np.random.permutation(folderlist) numbatches = len(t)//batchsize for batch in range(numbatches):\nbatchdata = np.zeros((batchsize,y,z,1))\nfor folder in range(batchsize): imgs = sourcepath+'/'+ t[folder + (batchbatchsize)][0]+'.jpg' image = io.imread(imgs).astype(np.float32) image = rgb2gray(image) image = resizeimage(image)\nimage = image.reshape(y,z,1)\nbatchdata[folder,:,:] = image yield batchdata\nbatchdata = np.zeros(((len(t)%batchsize,y,z,1))) #(y,z) is the final size of the input images\nfor folder in range(len(t)%batchsize): imgs = sourcepath+'/'+ t[folder + (batchbatchsize)][0]+'.jpg' image = io.imread(imgs).astype(np.float32) image = rgb2gray(image) image = resizeimage(image)\nimage = image.reshape(y,z,1)\nbatch_data[folder,:,:] = image
\n\nnumtestsequences = len(test)\nprint('# test sequences =', numtestsequences)
\n\nif (numtestsequences%batchsize) == 0: teststeps = int(numtestsequences/batchsize) else: teststeps = (numtestsequences//batch_size) + 1
\n\npredictions = model.predictgenerator(testgen,steps=test_steps)
\n\nWhile training even if it involves back propogation , it completed in around 25 min. prediction should not take much time. Not sure If I am doing anything wrong.
", "votes": 0, "replies": [] } ] }, { "id": 961705, "author_name": "", "author_url": "", "post_date": "2020-08-07T12:38:15.560000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "959496": "Are you running on Kaggle Notebooks or on your local machine? Also are you using GPUs or TPUs?\n\nIf you are not using TPUs I really recommend you to try them, they accelerate the training a lot!", "959481": "I have build CNN model with keras and I am using prediction_generator to predict on test data but prediction time is too long. It almost took 7 hrs to finish whole test set. I am doing resize and normalization of image before doing prediction and using 256*256 size as input. how we can improve on prediction time?", "961705": "" } }