{
  "id": 159367,
  "title": "UnavailableError: Socket closed",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/159367",
  "author_name": "",
  "post_date": "2020-06-17T08:30:06.462818800Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hi,\nI am getting this error. The code is copied from public notebooks for efficient nets.\nWhat I need is to run \"predict\" on training data:</p>\n\n<p>```\ndef get_train_dataset_1(ordered=True):\n    dataset = load_dataset(TRAINING_FILENAMES, labeled=True, ordered=ordered)\n    dataset = dataset.batch(BATCH_SIZE)\n    dataset = dataset.cache()\n    dataset = dataset.prefetch(AUTO) # prefetch next batch while training (autotune prefetch buffer size)\n    return dataset</p>\n\n<h1>Run new_b5_imagenet on train_dataset and save output</h1>\n\n<p>print('Computing new_b5_imagenet predictions...')</p>\n\n<p>train_dataset_1 = get_train_dataset_1(ordered=True)</p>\n\n<p>train_images_ds = train_dataset_1.map(lambda image, idnum: image)\nprobabilities = model_new_b5_imagenet.predict(train_images_ds)</p>\n\n<p>print(\"done\")\n```</p>\n\n<p>And I am getting this:</p>\n\n<blockquote>\n  <p></p><hr><p></p>\n  \n  <p>UnavailableError                          Traceback (most recent call last)\n   in \n  ----&gt; 1 probabilities = model_new_b5_imagenet.predict(train_images_ds)</p>\n</blockquote>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in _method_wrapper(self, *args, **kwargs)\n     85       raise ValueError('{} is not supported in multi-worker mode.'.format(\n     86           method.<strong>name</strong>))\n---&gt; 87     return method(self, *args, **kwargs)\n     88 \n     89   return tf_decorator.make_decorator(</p>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in predict(self, x, batch_size, verbose, steps, callbacks, max_queue_size, workers, use_multiprocessing)\n   1219       callbacks.on_predict_end()\n   1220     all_outputs = nest.map_structure_up_to(batch_outputs, concat, outputs)\n-&gt; 1221     return tf_utils.to_numpy_or_python_type(all_outputs)\n   1222 \n   1223   def reset_metrics(self):</p>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py in to_numpy_or_python_type(tensors)\n    521     return t  # Don't turn ragged or sparse tensors to NumPy.\n    522 \n--&gt; 523   return nest.map_structure(_to_single_numpy_or_python_type, tensors)\n    524 </p>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/nest.py in map_structure(func, *structure, **kwargs)\n    615 \n    616   return pack_sequence_as(\n--&gt; 617       structure[0], [func(*x) for x in entries],\n    618       expand_composites=expand_composites)\n    619 </p>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/nest.py in (.0)\n    615 \n    616   return pack_sequence_as(\n--&gt; 617       structure[0], [func(*x) for x in entries],\n    618       expand_composites=expand_composites)\n    619 </p>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py in _to_single_numpy_or_python_type(t)\n    517   def _to_single_numpy_or_python_type(t):\n    518     if isinstance(t, ops.Tensor):\n--&gt; 519       x = t.numpy()\n    520       return x.item() if np.ndim(x) == 0 else x\n    521     return t  # Don't turn ragged or sparse tensors to NumPy.</p>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py in numpy(self)\n    959     \"\"\"\n    960     # TODO(slebedev): Consider avoiding a copy for non-CPU or remote tensors.\n--&gt; 961     maybe_arr = self._numpy()  # pylint: disable=protected-access\n    962     return maybe_arr.copy() if isinstance(maybe_arr, np.ndarray) else maybe_arr\n    963 </p>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py in _numpy(self)\n    927       return self._numpy_internal()\n    928     except core._NotOkStatusException as e:\n--&gt; 929       six.raise_from(core._status_to_exception(e.code, e.message), None)\n    930 \n    931   @property</p>\n\n<p>/opt/conda/lib/python3.7/site-packages/six.py in raise_from(value, from_value)</p>\n\n<p>UnavailableError: Socket closed\nAdditional GRPC error information:\n{\"created\":\"@1592382265.576988450\",\"description\":\"Error received from peer ipv4:10.0.0.2:8470\",\"file\":\"external/com_github_grpc_grpc/src/core/lib/surface/call.cc\",\"file_line\":1056,\"grpc_message\":\"Socket closed\",\"grpc_status\":14}</p>",
  "messages": [
    {
      "id": "889971",
      "postDate": "06/17/2020 08:30:06",
      "content": "<p>Hi,\nI am getting this error. The code is copied from public notebooks for efficient nets.\nWhat I need is to run \"predict\" on training data:</p>\n\n<p>```\ndef get_train_dataset_1(ordered=True):\n    dataset = load_dataset(TRAINING_FILENAMES, labeled=True, ordered=ordered)\n    dataset = dataset.batch(BATCH_SIZE)\n    dataset = dataset.cache()\n    dataset = dataset.prefetch(AUTO) # prefetch next batch while training (autotune prefetch buffer size)\n    return dataset</p>\n\n<h1>Run new_b5_imagenet on train_dataset and save output</h1>\n\n<p>print('Computing new_b5_imagenet predictions...')</p>\n\n<p>train_dataset_1 = get_train_dataset_1(ordered=True)</p>\n\n<p>train_images_ds = train_dataset_1.map(lambda image, idnum: image)\nprobabilities = model_new_b5_imagenet.predict(train_images_ds)</p>\n\n<p>print(\"done\")\n```</p>\n\n<p>And I am getting this:</p>\n\n<blockquote>\n  <p></p><hr><p></p>\n  \n  <p>UnavailableError                          Traceback (most recent call last)\n   in \n  ----&gt; 1 probabilities = model_new_b5_imagenet.predict(train_images_ds)</p>\n</blockquote>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in _method_wrapper(self, *args, **kwargs)\n     85       raise ValueError('{} is not supported in multi-worker mode.'.format(\n     86           method.<strong>name</strong>))\n---&gt; 87     return method(self, *args, **kwargs)\n     88 \n     89   return tf_decorator.make_decorator(</p>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in predict(self, x, batch_size, verbose, steps, callbacks, max_queue_size, workers, use_multiprocessing)\n   1219       callbacks.on_predict_end()\n   1220     all_outputs = nest.map_structure_up_to(batch_outputs, concat, outputs)\n-&gt; 1221     return tf_utils.to_numpy_or_python_type(all_outputs)\n   1222 \n   1223   def reset_metrics(self):</p>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py in to_numpy_or_python_type(tensors)\n    521     return t  # Don't turn ragged or sparse tensors to NumPy.\n    522 \n--&gt; 523   return nest.map_structure(_to_single_numpy_or_python_type, tensors)\n    524 </p>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/nest.py in map_structure(func, *structure, **kwargs)\n    615 \n    616   return pack_sequence_as(\n--&gt; 617       structure[0], [func(*x) for x in entries],\n    618       expand_composites=expand_composites)\n    619 </p>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/nest.py in (.0)\n    615 \n    616   return pack_sequence_as(\n--&gt; 617       structure[0], [func(*x) for x in entries],\n    618       expand_composites=expand_composites)\n    619 </p>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py in _to_single_numpy_or_python_type(t)\n    517   def _to_single_numpy_or_python_type(t):\n    518     if isinstance(t, ops.Tensor):\n--&gt; 519       x = t.numpy()\n    520       return x.item() if np.ndim(x) == 0 else x\n    521     return t  # Don't turn ragged or sparse tensors to NumPy.</p>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py in numpy(self)\n    959     \"\"\"\n    960     # TODO(slebedev): Consider avoiding a copy for non-CPU or remote tensors.\n--&gt; 961     maybe_arr = self._numpy()  # pylint: disable=protected-access\n    962     return maybe_arr.copy() if isinstance(maybe_arr, np.ndarray) else maybe_arr\n    963 </p>\n\n<p>/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py in _numpy(self)\n    927       return self._numpy_internal()\n    928     except core._NotOkStatusException as e:\n--&gt; 929       six.raise_from(core._status_to_exception(e.code, e.message), None)\n    930 \n    931   @property</p>\n\n<p>/opt/conda/lib/python3.7/site-packages/six.py in raise_from(value, from_value)</p>\n\n<p>UnavailableError: Socket closed\nAdditional GRPC error information:\n{\"created\":\"@1592382265.576988450\",\"description\":\"Error received from peer ipv4:10.0.0.2:8470\",\"file\":\"external/com_github_grpc_grpc/src/core/lib/surface/call.cc\",\"file_line\":1056,\"grpc_message\":\"Socket closed\",\"grpc_status\":14}</p>",
      "rawMarkdown": "Hi,\nI am getting this error. The code is copied from public notebooks for efficient nets.\nWhat I need is to run \"predict\" on training data:\n\n```\ndef get_train_dataset_1(ordered=True):\n    dataset = load_dataset(TRAINING_FILENAMES, labeled=True, ordered=ordered)\n    dataset = dataset.batch(BATCH_SIZE)\n    dataset = dataset.cache()\n    dataset = dataset.prefetch(AUTO) # prefetch next batch while training (autotune prefetch buffer size)\n    return dataset\n\n# Run new_b5_imagenet on train_dataset and save output\n\nprint('Computing new_b5_imagenet predictions...')\n\ntrain_dataset_1 = get_train_dataset_1(ordered=True)\n\ntrain_images_ds = train_dataset_1.map(lambda image, idnum: image)\nprobabilities = model_new_b5_imagenet.predict(train_images_ds)\n\nprint(\"done\")\n```\n\nAnd I am getting this:\n\n\n\n\n\n&gt; ---------------------------------------------------------------------------\nUnavailableError                          Traceback (most recent call last)",
      "votes": null
    },
    {
      "id": "889972",
      "postDate": "06/17/2020 08:31:27",
      "content": "<p>P.S. Checked this (<a href=\"https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/discussion/139586\">https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/discussion/139586</a>). Doesn't help.</p>",
      "rawMarkdown": "P.S. Checked this (https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/discussion/139586). Doesn't help.",
      "votes": null
    },
    {
      "id": "890162",
      "postDate": "06/17/2020 10:57:24",
      "content": "<p>Same here. It was raised when I tried to implement lookahead on TPU. \n(without lookahead everything works fine)</p>",
      "rawMarkdown": "Same here. It was raised when I tried to implement lookahead on TPU. \n(without lookahead everything works fine)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 889972,
      "author_name": "fizpok",
      "author_url": "",
      "post_date": "06/17/2020 08:31:27",
      "content": "<p>P.S. Checked this (<a href=\"https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/discussion/139586\">https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/discussion/139586</a>). Doesn't help.</p>",
      "votes": null,
      "replies": [
        {
          "id": 890162,
          "author_name": "stanislavblinov",
          "author_url": "",
          "post_date": "06/17/2020 10:57:24",
          "content": "<p>Same here. It was raised when I tried to implement lookahead on TPU. \n(without lookahead everything works fine)</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "889971": "Hi,\nI am getting this error. The code is copied from public notebooks for efficient nets.\nWhat I need is to run \"predict\" on training data:\n\n```\ndef get_train_dataset_1(ordered=True):\n    dataset = load_dataset(TRAINING_FILENAMES, labeled=True, ordered=ordered)\n    dataset = dataset.batch(BATCH_SIZE)\n    dataset = dataset.cache()\n    dataset = dataset.prefetch(AUTO) # prefetch next batch while training (autotune prefetch buffer size)\n    return dataset\n\n# Run new_b5_imagenet on train_dataset and save output\n\nprint('Computing new_b5_imagenet predictions...')\n\ntrain_dataset_1 = get_train_dataset_1(ordered=True)\n\ntrain_images_ds = train_dataset_1.map(lambda image, idnum: image)\nprobabilities = model_new_b5_imagenet.predict(train_images_ds)\n\nprint(\"done\")\n```\n\nAnd I am getting this:\n\n\n\n\n\n&gt; ---------------------------------------------------------------------------\nUnavailableError                          Traceback (most recent call last)",
    "889972": "P.S. Checked this (https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/discussion/139586). Doesn't help.",
    "890162": "Same here. It was raised when I tried to implement lookahead on TPU. \n(without lookahead everything works fine)"
  },
  "source": "meta"
}