Hi, i am trying to augment images using ImageGenerator. However, fit_generator method is throwing error on TPU.\n\nNotFoundError: {{function_node __inference_train_function_662461}} No registered 'PyFunc' OpKernel for 'CPU' devices compatible with node {{node PyFunc}}\n . Registered: <no registered kernels>\n\nPlease help.
@cdeotte has created two excellent notebooks for using augmentations with TPUs that should help get you started!\n- How To: Rotation Augmentation\n- How To: CutMix MixUP
", "rawMarkdown": "@cdeotte has created two excellent notebooks for using augmentations with TPUs that should help get you started!\n- [How To: Rotation Augmentation](https://www.kaggle.com/c/flower-classification-with-tpus/discussion/132191)\n- [How To: CutMix MixUP](https://www.kaggle.com/c/flower-classification-with-tpus/discussion/132935)", "votes": 3, "replies": [ { "id": 815599, "postDate": "2020-04-21T17:06:08.080Z", "content": "Thanks! Will surely look .
", "rawMarkdown": "Thanks! Will surely look ." } ] }, { "id": 812581, "postDate": "2020-04-18T20:34:28.467Z", "content": "Hi, i am trying to augment images using ImageGenerator. However, fit_generator method is throwing error on TPU.\n\nNotFoundError: {{function_node __inference_train_function_662461}} No registered 'PyFunc' OpKernel for 'CPU' devices compatible with node {{node PyFunc}}\n . Registered: <no registered kernels>\n\nPlease help.
@jessemostipak I got the same error as above but I don't want to do data augmentation but when I try to load the images using imagedatagenrator and try to use TPU I can't do it and getting the same error. Please help
", "rawMarkdown": "@jessemostipak I got the same error as above but I don't want to do data augmentation but when I try to load the images using imagedatagenrator and try to use TPU I can't do it and getting the same error. Please help", "votes": 1, "replies": [ { "id": 823306, "postDate": "2020-04-27T14:51:57.160Z", "content": "it's difficult to tell what's happening without seeing your code, but I can recommend following the Getting Started notebook for this competition - everything in here works, and this notebook is a great way to get up and running with TPUs and the Flower Classification dataset.
", "rawMarkdown": "it's difficult to tell what's happening without seeing your code, but I can recommend following the [Getting Started notebook for this competition](https://www.kaggle.com/mgornergoogle/getting-started-with-100-flowers-on-tpu/) - everything in here works, and this notebook is a great way to get up and running with TPUs and the Flower Classification dataset." }, { "id": 823692, "postDate": "2020-04-27T20:07:51.537Z", "content": "The ImageDataGenerator utility from Keras will not work on TPUs. Load your images normally through the tf.data.Dataset API as in the TPU starter notebook.
", "rawMarkdown": "The ImageDataGenerator utility from Keras will not work on TPUs. Load your images normally through the tf.data.Dataset API as in the [TPU starter notebook](https://www.kaggle.com/mgornergoogle/getting-started-with-100-flowers-on-tpu/)." } ] }, { "id": 828948, "postDate": "2020-05-01T12:04:48.010Z", "content": "@jessemostipak Okk if the notebook doesn't open (try copy and edit option then you can see full code), but can you suggest me a way where I can load 100000 images normal RGB format and train that model on TPU.
", "rawMarkdown": "@jessemostipak Okk if the notebook doesn't open (try copy and edit option then you can see full code), but can you suggest me a way where I can load 100000 images normal RGB format and train that model on TPU." }, { "id": 825747, "postDate": "2020-04-29T08:07:08.233Z", "content": "@jessemostipak @mgornergoogle As you ask here is my notebook where I want to train my model with TPU.\nPlease do whatever you want to but help me so that I can train my model with TPU.\nhttps://www.kaggle.com/chirag003/video-classification-normal-tpu/edit/run/30353251
", "rawMarkdown": "@jessemostipak @mgornergoogle As you ask here is my notebook where I want to train my model with TPU.\nPlease do whatever you want to but help me so that I can train my model with TPU.\nhttps://www.kaggle.com/chirag003/video-classification-normal-tpu/edit/run/30353251", "replies": [ { "id": 828010, "postDate": "2020-04-30T17:47:51.497Z", "content": "thanks for sharing the notebook! 3unfortunately when I try to open it it freezes and I can't see anything beyond the first code block. as Martin mentioned in an earlier reply, do make sure that you're loading images through the tf.data.Dataset API!
", "rawMarkdown": "thanks for sharing the notebook! 3unfortunately when I try to open it it freezes and I can't see anything beyond the first code block. as Martin mentioned in an earlier reply, do make sure that you're loading images through the tf.data.Dataset API!" } ] }, { "id": 812736, "postDate": "2020-04-19T02:59:57.827Z", "content": "@pawanverma did you mean keras ImageGenerator?
\n\nAugmentation libraries are not by default supported in TPU
", "rawMarkdown": "@pawanverma did you mean keras ImageGenerator?\n\nAugmentation libraries are not by default supported in TPU", "replies": [ { "id": 812980, "postDate": "2020-04-19T09:36:13.813Z", "content": "Yes, i was trying to run keras ImageGenerator. Any other method to generate on the fly augmentation
", "rawMarkdown": "Yes, i was trying to run keras ImageGenerator. Any other method to generate on the fly augmentation" } ] }, { "id": 832307, "postDate": "2020-05-04T03:27:36.403Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 815583, "author_name": "Jesse Mostipak", "author_url": "", "post_date": "2020-04-21T16:50:36.610000", "content": "@cdeotte has created two excellent notebooks for using augmentations with TPUs that should help get you started!\n- How To: Rotation Augmentation\n- How To: CutMix MixUP
", "votes": 3, "replies": [ { "id": 815599, "author_name": "pawan verma", "author_url": "", "post_date": "2020-04-21T17:06:08.080000", "content": "Thanks! Will surely look .
", "votes": 0, "replies": [] } ] }, { "id": 820277, "author_name": "Chirag", "author_url": "", "post_date": "2020-04-25T09:42:51.003000", "content": "@jessemostipak I got the same error as above but I don't want to do data augmentation but when I try to load the images using imagedatagenrator and try to use TPU I can't do it and getting the same error. Please help
", "votes": 1, "replies": [ { "id": 823306, "author_name": "Jesse Mostipak", "author_url": "", "post_date": "2020-04-27T14:51:57.160000", "content": "it's difficult to tell what's happening without seeing your code, but I can recommend following the Getting Started notebook for this competition - everything in here works, and this notebook is a great way to get up and running with TPUs and the Flower Classification dataset.
", "votes": 0, "replies": [] }, { "id": 823692, "author_name": "Martin Görner", "author_url": "", "post_date": "2020-04-27T20:07:51.537000", "content": "The ImageDataGenerator utility from Keras will not work on TPUs. Load your images normally through the tf.data.Dataset API as in the TPU starter notebook.
", "votes": 0, "replies": [] } ] }, { "id": 828948, "author_name": "Chirag", "author_url": "", "post_date": "2020-05-01T12:04:48.010000", "content": "@jessemostipak Okk if the notebook doesn't open (try copy and edit option then you can see full code), but can you suggest me a way where I can load 100000 images normal RGB format and train that model on TPU.
", "votes": 0, "replies": [] }, { "id": 825747, "author_name": "Chirag", "author_url": "", "post_date": "2020-04-29T08:07:08.233000", "content": "@jessemostipak @mgornergoogle As you ask here is my notebook where I want to train my model with TPU.\nPlease do whatever you want to but help me so that I can train my model with TPU.\nhttps://www.kaggle.com/chirag003/video-classification-normal-tpu/edit/run/30353251
", "votes": 0, "replies": [ { "id": 828010, "author_name": "Jesse Mostipak", "author_url": "", "post_date": "2020-04-30T17:47:51.497000", "content": "thanks for sharing the notebook! 3unfortunately when I try to open it it freezes and I can't see anything beyond the first code block. as Martin mentioned in an earlier reply, do make sure that you're loading images through the tf.data.Dataset API!
", "votes": 0, "replies": [] } ] }, { "id": 812736, "author_name": "Kurian Benoy", "author_url": "", "post_date": "2020-04-19T02:59:57.827000", "content": "@pawanverma did you mean keras ImageGenerator?
\n\nAugmentation libraries are not by default supported in TPU
", "votes": 0, "replies": [ { "id": 812980, "author_name": "pawan verma", "author_url": "", "post_date": "2020-04-19T09:36:13.813000", "content": "Yes, i was trying to run keras ImageGenerator. Any other method to generate on the fly augmentation
", "votes": 0, "replies": [] } ] }, { "id": 832307, "author_name": "", "author_url": "", "post_date": "2020-05-04T03:27:36.403000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "815583": "@cdeotte has created two excellent notebooks for using augmentations with TPUs that should help get you started!\n- [How To: Rotation Augmentation](https://www.kaggle.com/c/flower-classification-with-tpus/discussion/132191)\n- [How To: CutMix MixUP](https://www.kaggle.com/c/flower-classification-with-tpus/discussion/132935)", "812581": "Hi, i am trying to augment images using ImageGenerator. However, fit_generator method is throwing error on TPU.\n```\nNotFoundError: {{function_node __inference_train_function_662461}} No registered 'PyFunc' OpKernel for 'CPU' devices compatible with node {{node PyFunc}}\n\t. Registered: ", "820277": "@jessemostipak I got the same error as above but I don't want to do data augmentation but when I try to load the images using imagedatagenrator and try to use TPU I can't do it and getting the same error. Please help", "828948": "@jessemostipak Okk if the notebook doesn't open (try copy and edit option then you can see full code), but can you suggest me a way where I can load 100000 images normal RGB format and train that model on TPU.", "825747": "@jessemostipak @mgornergoogle As you ask here is my notebook where I want to train my model with TPU.\nPlease do whatever you want to but help me so that I can train my model with TPU.\nhttps://www.kaggle.com/chirag003/video-classification-normal-tpu/edit/run/30353251", "812736": "@pawanverma did you mean keras ImageGenerator?\n\nAugmentation libraries are not by default supported in TPU", "832307": "" } }