Hi everyone,
\nI put some work into this competition early on, but I ended up dropping out and focusing more on developing a tensorflow2.x native implementation of CLAHE, which I was surprised to find didn't have tensorflow support yet.
I've lost track of the latest developments here and I'm not sure if any of you are even using CLAHE in your pipelines, but I figured I'd drop a link to my work in case anyone is interested in test-driving it.
\nI just published everything on pypi as the pip-installable tf_clahe package and the source code is available on github.
I've also submitted this feature as a PR to tensorflow addons, so maybe in the near future we'll have an even easier-to-use tfa.image.clahe() function.
Feel free to open issues for bugs/feature requests!
", "messages": [ { "id": "1195306", "postDate": "02/10/2021 17:16:07", "content": "Hi everyone,
\nI put some work into this competition early on, but I ended up dropping out and focusing more on developing a tensorflow2.x native implementation of CLAHE, which I was surprised to find didn't have tensorflow support yet.
I've lost track of the latest developments here and I'm not sure if any of you are even using CLAHE in your pipelines, but I figured I'd drop a link to my work in case anyone is interested in test-driving it.
\nI just published everything on pypi as the pip-installable tf_clahe package and the source code is available on github.
I've also submitted this feature as a PR to tensorflow addons, so maybe in the near future we'll have an even easier-to-use tfa.image.clahe() function.
Feel free to open issues for bugs/feature requests!
", "rawMarkdown": "Hi everyone,\nI put some work into this competition early on, but I ended up dropping out and focusing more on developing a tensorflow2.x native implementation of CLAHE, which I was surprised to find didn't have tensorflow support yet.\n\nI've lost track of the latest developments here and I'm not sure if any of you are even using CLAHE in your pipelines, but I figured I'd drop a link to my work in case anyone is interested in test-driving it.\n\nI just published everything on [pypi](https://pypi.org/project/tf-clahe/) as the pip-installable `tf_clahe` package and the source code is available [on github](https://github.com/isears/tf_clahe).\n\nI've also submitted this feature as a [PR](https://github.com/tensorflow/addons/pull/2362) to tensorflow addons, so maybe in the near future we'll have an even easier-to-use `tfa.image.clahe()` function.\n\nFeel free to open issues for bugs/feature requests!", "votes": null }, { "id": "1198499", "postDate": "02/13/2021 05:47:14", "content": "Thanks a lot for your sharing. I'll try to use!
", "rawMarkdown": "Thanks a lot for your sharing. I'll try to use!", "votes": null }, { "id": "1210464", "postDate": "02/19/2021 13:04:04", "content": "Thanks for this!. I'm just curious, (please correct me if I'm wrong), if i understand it right, CLAHE application requires 2 hyperparameters which are (a) number of tiles and (b) clip limit. How did you set those parameters? Are they fixed?
", "rawMarkdown": "Thanks for this!. I'm just curious, (please correct me if I'm wrong), if i understand it right, CLAHE application requires 2 hyperparameters which are (a) number of tiles and (b) clip limit. How did you set those parameters? Are they fixed?", "votes": null }, { "id": "1210613", "postDate": "02/19/2021 14:58:57", "content": "Yeah, you're right. The function I've written assumes sane defaults for a chest X-ray (8x8 tiling and 4.0 clip limit), but you can also specify other values using the tile_grid_size and clip_limit params:
\ntf_clahe.clahe(img, tile_grid_size=(4,4), clip_limit=3.0)
I'll update the README to better document that.
", "rawMarkdown": "Yeah, you're right. The function I've written assumes sane defaults for a chest X-ray (8x8 tiling and 4.0 clip limit), but you can also specify other values using the tile_grid_size and clip_limit params:\n\n`tf_clahe.clahe(img, tile_grid_size=(4,4), clip_limit=3.0)`\n\nI'll update the README to better document that.", "votes": null }, { "id": "1211704", "postDate": "02/20/2021 13:14:36", "content": "Thanks for clarifying :)
", "rawMarkdown": "Thanks for clarifying :)", "votes": null }, { "id": "1213190", "postDate": "02/22/2021 00:06:53", "content": "it doesnt work with TPU. Could you share your example that is correctly worked in TPU.
", "rawMarkdown": "it doesnt work with TPU. Could you share your example that is correctly worked in TPU.", "votes": null }, { "id": "1236619", "postDate": "03/13/2021 10:06:32", "content": "Hi @isaacsears ,
\nThank you for this tool.
I am having difficulty using it with GPU and TUP.
\nI am using this code as a pipeline.
\nhttps://www.kaggle.com/maksymshkliarevskyi/ranzcr-xception-tpu-baseline
I want to use tf_clahe for all mages.
\nwhen I added to This part of the code.
\nimport tf_clahe\nimport tensorflow as tf\n\n@tf.function(experimental_compile=True) # Enable XLA\ndef fast_clahe(img):\n return tf_clahe.clahe(img, gpu_optimized=True)\n file_bytes = tf.io.read_file(path)\n img = tf.image.decode_jpeg(file_bytes, channels = 3)\n img = fast_clahe(img)\n img = tf.cast(img, tf.float32) / 255.0\n img = tf.image.resize(img, target_size)\nI got this error
\nInvalidArgumentError: 2 root error(s) found.\n (0) Invalid argument: Function invoked by the following node is not compilable: {{node PartitionedCall}} = PartitionedCall[Tin=[DT_UINT8], Tout=[DT_UINT8], _XlaHasReferenceVars=false, _XlaMustCompile=true, _collective_manager_ids=[], _input_hostmem=[], _read_only_resource_inputs=[], config=\"\", config_proto=\"\\n\\007\\n\\003CPU\\020\\001\\n\\007\\n\\003GPU\\020\\0012\\005*\\0010J\\0008\\001\\202\\001\\000\", executor_type=\"\", f=__inference_fast_clahe_397[], _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](DecodeJpeg).\nUncompilable nodes:\nPartitionedCall: could not instantiate call: '__inference_fast_clahe_397'\n Stacktrace:\n Node: PartitionedCall, function: \n\n [[PartitionedCall]]\n [[IteratorGetNext]]\n (1) Invalid argument: Function invoked by the following node is not compilable: {{node PartitionedCall}} = PartitionedCall[Tin=[DT_UINT8], Tout=[DT_UINT8], _XlaHasReferenceVars=false, _XlaMustCompile=true, _collective_manager_ids=[], _input_hostmem=[], _read_only_resource_inputs=[], config=\"\", config_proto=\"\\n\\007\\n\\003CPU\\020\\001\\n\\007\\n\\003GPU\\020\\0012\\005*\\0010J\\0008\\001\\202\\001\\000\", executor_type=\"\", f=__inference_fast_clahe_397[], _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](DecodeJpeg).\nUncompilable nodes:\nPartitionedCall: could not instantiate call: '__inference_fast_clahe_397'\n Stacktrace:\n Node: PartitionedCall, function: \n\n [[PartitionedCall]]\n [[IteratorGetNext]]\n [[assert_less_equal/Assert/AssertGuard/else/_11/assert_less_equal/Assert/AssertGuard/Assert/data_2/_92]]\n0 successful operations.\n0 derived errors ignored. [Op:__inference_train_function_10702]\n\nFunction call stack:\ntrain_function -> train_function\nHi Faisal,
\nI think this issue is related to using clahe on the TPU. I haven't gotten around to figuring out why it's not working on the TPU, but I'll open an issue for this on the repo and update it if I make progress.
\nAlso, keep me posted if you find any workarounds.
", "rawMarkdown": "Hi Faisal,\n\nI think this issue is related to using clahe on the TPU. I haven't gotten around to figuring out why it's not working on the TPU, but I'll open an issue for this on the repo and update it if I make progress.\n\nAlso, keep me posted if you find any workarounds.", "votes": null }, { "id": "1236826", "postDate": "03/13/2021 13:42:02", "content": "Yes, sure I will.
\nbut even with GPU not working.
\nit will be great if you make a Kaggle notebook with examples
\nI suggest you have a look at tf_sprinkles.
\nit is working well with TPU and GPU.
\nhttps://github.com/Engineero/tf_sprinkles
Thank you for your efforts tfa.image.clahe()will useful for AI healthcare.
Thanks a lot for your sharing. I'll try to use!
", "votes": null, "replies": [] }, { "id": 1210464, "author_name": "djbacad", "author_url": "", "post_date": "02/19/2021 13:04:04", "content": "Thanks for this!. I'm just curious, (please correct me if I'm wrong), if i understand it right, CLAHE application requires 2 hyperparameters which are (a) number of tiles and (b) clip limit. How did you set those parameters? Are they fixed?
", "votes": null, "replies": [ { "id": 1210613, "author_name": "isaacsears", "author_url": "", "post_date": "02/19/2021 14:58:57", "content": "Yeah, you're right. The function I've written assumes sane defaults for a chest X-ray (8x8 tiling and 4.0 clip limit), but you can also specify other values using the tile_grid_size and clip_limit params:
\ntf_clahe.clahe(img, tile_grid_size=(4,4), clip_limit=3.0)
I'll update the README to better document that.
", "votes": null, "replies": [] }, { "id": 1211704, "author_name": "djbacad", "author_url": "", "post_date": "02/20/2021 13:14:36", "content": "Thanks for clarifying :)
", "votes": null, "replies": [] } ] }, { "id": 1213190, "author_name": "sahini", "author_url": "", "post_date": "02/22/2021 00:06:53", "content": "it doesnt work with TPU. Could you share your example that is correctly worked in TPU.
", "votes": null, "replies": [] }, { "id": 1236619, "author_name": "faisalalsrheed", "author_url": "", "post_date": "03/13/2021 10:06:32", "content": "Hi @isaacsears ,
\nThank you for this tool.
I am having difficulty using it with GPU and TUP.
\nI am using this code as a pipeline.
\nhttps://www.kaggle.com/maksymshkliarevskyi/ranzcr-xception-tpu-baseline
I want to use tf_clahe for all mages.
\nwhen I added to This part of the code.
\nimport tf_clahe\nimport tensorflow as tf\n\n@tf.function(experimental_compile=True) # Enable XLA\ndef fast_clahe(img):\n return tf_clahe.clahe(img, gpu_optimized=True)\n file_bytes = tf.io.read_file(path)\n img = tf.image.decode_jpeg(file_bytes, channels = 3)\n img = fast_clahe(img)\n img = tf.cast(img, tf.float32) / 255.0\n img = tf.image.resize(img, target_size)\nI got this error
\nInvalidArgumentError: 2 root error(s) found.\n (0) Invalid argument: Function invoked by the following node is not compilable: {{node PartitionedCall}} = PartitionedCall[Tin=[DT_UINT8], Tout=[DT_UINT8], _XlaHasReferenceVars=false, _XlaMustCompile=true, _collective_manager_ids=[], _input_hostmem=[], _read_only_resource_inputs=[], config=\"\", config_proto=\"\\n\\007\\n\\003CPU\\020\\001\\n\\007\\n\\003GPU\\020\\0012\\005*\\0010J\\0008\\001\\202\\001\\000\", executor_type=\"\", f=__inference_fast_clahe_397[], _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](DecodeJpeg).\nUncompilable nodes:\nPartitionedCall: could not instantiate call: '__inference_fast_clahe_397'\n Stacktrace:\n Node: PartitionedCall, function: \n\n [[PartitionedCall]]\n [[IteratorGetNext]]\n (1) Invalid argument: Function invoked by the following node is not compilable: {{node PartitionedCall}} = PartitionedCall[Tin=[DT_UINT8], Tout=[DT_UINT8], _XlaHasReferenceVars=false, _XlaMustCompile=true, _collective_manager_ids=[], _input_hostmem=[], _read_only_resource_inputs=[], config=\"\", config_proto=\"\\n\\007\\n\\003CPU\\020\\001\\n\\007\\n\\003GPU\\020\\0012\\005*\\0010J\\0008\\001\\202\\001\\000\", executor_type=\"\", f=__inference_fast_clahe_397[], _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](DecodeJpeg).\nUncompilable nodes:\nPartitionedCall: could not instantiate call: '__inference_fast_clahe_397'\n Stacktrace:\n Node: PartitionedCall, function: \n\n [[PartitionedCall]]\n [[IteratorGetNext]]\n [[assert_less_equal/Assert/AssertGuard/else/_11/assert_less_equal/Assert/AssertGuard/Assert/data_2/_92]]\n0 successful operations.\n0 derived errors ignored. [Op:__inference_train_function_10702]\n\nFunction call stack:\ntrain_function -> train_function\nHi Faisal,
\nI think this issue is related to using clahe on the TPU. I haven't gotten around to figuring out why it's not working on the TPU, but I'll open an issue for this on the repo and update it if I make progress.
\nAlso, keep me posted if you find any workarounds.
", "votes": null, "replies": [] }, { "id": 1236826, "author_name": "faisalalsrheed", "author_url": "", "post_date": "03/13/2021 13:42:02", "content": "Yes, sure I will.
\nbut even with GPU not working.
\nit will be great if you make a Kaggle notebook with examples
\nI suggest you have a look at tf_sprinkles.
\nit is working well with TPU and GPU.
\nhttps://github.com/Engineero/tf_sprinkles
Thank you for your efforts tfa.image.clahe()will useful for AI healthcare.