{
  "id": 168316,
  "title": "Can someone help me with this error ?",
  "url": "/competitions/alaska2-image-steganalysis/discussion/168316",
  "author_name": "",
  "post_date": "2020-07-20T06:13:40.221591300Z",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>UnavailableError: 6 root error(s) found.\n  (0) Unavailable: {{function_node __inference_train_function_136689}} failed to connect to all addresses\nAdditional GRPC error information:\n{\"created\":\"@1595225310.568758019\",\"description\":\"Failed to pick subchannel\",\"file\":\"third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc\",\"file_line\":3959,\"referenced_errors\":[{\"created\":\"@1595225310.556742662\",\"description\":\"failed to connect to all addresses\",\"file\":\"third_party/grpc/src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc\",\"file_line\":394,\"grpc_status\":14}]}\n     [[{{node MultiDeviceIteratorGetNextFromShard}}]]\n     [[RemoteCall]]\n     [[IteratorGetNextAsOptional_5]]\n  (1) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (2) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (3) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (4) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (5) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n0 successful operations.\n3 derived errors ignored.</p>",
  "messages": [
    {
      "id": "936290",
      "postDate": "07/20/2020 06:13:40",
      "content": "<p>UnavailableError: 6 root error(s) found.\n  (0) Unavailable: {{function_node __inference_train_function_136689}} failed to connect to all addresses\nAdditional GRPC error information:\n{\"created\":\"@1595225310.568758019\",\"description\":\"Failed to pick subchannel\",\"file\":\"third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc\",\"file_line\":3959,\"referenced_errors\":[{\"created\":\"@1595225310.556742662\",\"description\":\"failed to connect to all addresses\",\"file\":\"third_party/grpc/src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc\",\"file_line\":394,\"grpc_status\":14}]}\n     [[{{node MultiDeviceIteratorGetNextFromShard}}]]\n     [[RemoteCall]]\n     [[IteratorGetNextAsOptional_5]]\n  (1) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (2) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (3) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (4) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (5) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n0 successful operations.\n3 derived errors ignored.</p>",
      "rawMarkdown": "UnavailableError: 6 root error(s) found.\n  (0) Unavailable: {{function_node __inference_train_function_136689}} failed to connect to all addresses\nAdditional GRPC error information:\n{\"created\":\"@1595225310.568758019\",\"description\":\"Failed to pick subchannel\",\"file\":\"third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc\",\"file_line\":3959,\"referenced_errors\":[{\"created\":\"@1595225310.556742662\",\"description\":\"failed to connect to all addresses\",\"file\":\"third_party/grpc/src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc\",\"file_line\":394,\"grpc_status\":14}]}\n\t [[{{node MultiDeviceIteratorGetNextFromShard}}]]\n\t [[RemoteCall]]\n\t [[IteratorGetNextAsOptional_5]]\n  (1) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (2) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (3) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (4) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (5) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n0 successful operations.\n3 derived errors ignored.",
      "votes": null
    },
    {
      "id": "936465",
      "postDate": "07/20/2020 08:34:38",
      "content": "<p>Are you using Gpu or Tpu? </p>",
      "rawMarkdown": "Are you using Gpu or Tpu?",
      "votes": null
    },
    {
      "id": "936470",
      "postDate": "07/20/2020 08:37:58",
      "content": "<p>yes i am using tpu</p>",
      "rawMarkdown": "yes i am using tpu",
      "votes": null
    },
    {
      "id": "936510",
      "postDate": "07/20/2020 09:26:49",
      "content": "<p>I definitely saw such error when I used TPU, I tried to train model using TF Dataset from_generator, TPU didn't understand how parallelize samples from generator. Unfortunately I didn't find solution and just rewrite my code to use simple TF Dataset from_tensor_slices</p>",
      "rawMarkdown": "I definitely saw such error when I used TPU, I tried to train model using TF Dataset from_generator, TPU didn't understand how parallelize samples from generator. Unfortunately I didn't find solution and just rewrite my code to use simple TF Dataset from_tensor_slices",
      "votes": null
    },
    {
      "id": "936518",
      "postDate": "07/20/2020 09:33:18",
      "content": "<p>`def decode_image(file,label=None,img_size=(img_size,img_size)) :</p>\n\n<ol>\n<li><p>func = tf.py_function(JPEGdecompressYCbCr,[file],[tf.float16,tf.float16,3])</p>\n\n<p>image = tf.image.decode_jpeg(func, channels = 0)</p>\n\n<p>image = tf.cast(image, tf.float16) /255.0`  </p></li>\n</ol>\n\n<p>actually in my code the error was due to the first line of the decode func that is \ntf.py_function() \ni did no0t find any solution so i changed. See if You can get any idea how to make it correct</p>",
      "rawMarkdown": "`def decode_image(file,label=None,img_size=(img_size,img_size)) :\n    \n    \n 1.    func = tf.py_function(JPEGdecompressYCbCr,[file],[tf.float16,tf.float16,3])\n\n    image = tf.image.decode_jpeg(func, channels = 0)\n\n    image = tf.cast(image, tf.float16) /255.0`  \n\n\n\nactually in my code the error was due to the first line of the decode func that is \ntf.py_function() \ni did no0t find any solution so i changed. See if You can get any idea how to make it correct",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 936465,
      "author_name": "aybatov",
      "author_url": "",
      "post_date": "07/20/2020 08:34:38",
      "content": "<p>Are you using Gpu or Tpu? </p>",
      "votes": null,
      "replies": [
        {
          "id": 936470,
          "author_name": "jaidevchittoria",
          "author_url": "",
          "post_date": "07/20/2020 08:37:58",
          "content": "<p>yes i am using tpu</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 936510,
          "author_name": "aybatov",
          "author_url": "",
          "post_date": "07/20/2020 09:26:49",
          "content": "<p>I definitely saw such error when I used TPU, I tried to train model using TF Dataset from_generator, TPU didn't understand how parallelize samples from generator. Unfortunately I didn't find solution and just rewrite my code to use simple TF Dataset from_tensor_slices</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 936518,
          "author_name": "jaidevchittoria",
          "author_url": "",
          "post_date": "07/20/2020 09:33:18",
          "content": "<p>`def decode_image(file,label=None,img_size=(img_size,img_size)) :</p>\n\n<ol>\n<li><p>func = tf.py_function(JPEGdecompressYCbCr,[file],[tf.float16,tf.float16,3])</p>\n\n<p>image = tf.image.decode_jpeg(func, channels = 0)</p>\n\n<p>image = tf.cast(image, tf.float16) /255.0`  </p></li>\n</ol>\n\n<p>actually in my code the error was due to the first line of the decode func that is \ntf.py_function() \ni did no0t find any solution so i changed. See if You can get any idea how to make it correct</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "936290": "UnavailableError: 6 root error(s) found.\n  (0) Unavailable: {{function_node __inference_train_function_136689}} failed to connect to all addresses\nAdditional GRPC error information:\n{\"created\":\"@1595225310.568758019\",\"description\":\"Failed to pick subchannel\",\"file\":\"third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc\",\"file_line\":3959,\"referenced_errors\":[{\"created\":\"@1595225310.556742662\",\"description\":\"failed to connect to all addresses\",\"file\":\"third_party/grpc/src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc\",\"file_line\":394,\"grpc_status\":14}]}\n\t [[{{node MultiDeviceIteratorGetNextFromShard}}]]\n\t [[RemoteCall]]\n\t [[IteratorGetNextAsOptional_5]]\n  (1) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (2) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (3) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (4) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n  (5) Cancelled: {{function_node __inference_train_function_136689}} Function was cancelled before it was started\n0 successful operations.\n3 derived errors ignored.",
    "936465": "Are you using Gpu or Tpu?",
    "936470": "yes i am using tpu",
    "936510": "I definitely saw such error when I used TPU, I tried to train model using TF Dataset from_generator, TPU didn't understand how parallelize samples from generator. Unfortunately I didn't find solution and just rewrite my code to use simple TF Dataset from_tensor_slices",
    "936518": "`def decode_image(file,label=None,img_size=(img_size,img_size)) :\n    \n    \n 1.    func = tf.py_function(JPEGdecompressYCbCr,[file],[tf.float16,tf.float16,3])\n\n    image = tf.image.decode_jpeg(func, channels = 0)\n\n    image = tf.cast(image, tf.float16) /255.0`  \n\n\n\nactually in my code the error was due to the first line of the decode func that is \ntf.py_function() \ni did no0t find any solution so i changed. See if You can get any idea how to make it correct"
  },
  "source": "meta"
}