Error seemingly comes out the blue
\n\nI'm trying to iterate on a .take() operation on a dataset I created and am getting this error:
\n\nFile system scheme '[local]' not implemented (file: '')
I can do data.take(3) let's say and it returns 3 tensors, but when I do something like:
for image, label in dataset.take(3):
I get that error
", "messages": [ { "id": "812639", "postDate": "04/18/2020 21:40:37", "content": "Error seemingly comes out the blue
\n\nI'm trying to iterate on a .take() operation on a dataset I created and am getting this error:
\n\nFile system scheme '[local]' not implemented (file: '')
I can do data.take(3) let's say and it returns 3 tensors, but when I do something like:
for image, label in dataset.take(3):
I get that error
", "rawMarkdown": "Error seemingly comes out the blue\n\nI'm trying to iterate on a .take() operation on a dataset I created and am getting this error:\n\n`File system scheme '[local]' not implemented (file: '')`\n\nI can do `data.take(3)` let's say and it returns 3 tensors, but when I do something like:\n\n`for image, label in dataset.take(3):`\n\nI get that error", "votes": null }, { "id": "814685", "postDate": "04/20/2020 21:33:00", "content": "The dataset has to be loaded from GCS if you want to use it on TPU.\nUse KaggleDatasets().get_gcs_path() to get a GCS location for the dataset. See kaggle.com/docs/tpu documentation.
", "rawMarkdown": "The dataset has to be loaded from GCS if you want to use it on TPU.\nUse KaggleDatasets().get_gcs_path() to get a GCS location for the dataset. See [kaggle.com/docs/tpu](https://www.kaggle.com/docs/tpu) documentation.", "votes": null }, { "id": "815936", "postDate": "04/22/2020 00:40:30", "content": "that is actually how im loading the dataset in, I believe this error came from loading a saved model (which I did by commit saving, downloading the output, then uploading it back into the kernel).
\n\nIll update this if I can reproduce it when I return to training with a saved model.
", "rawMarkdown": "that is actually how im loading the dataset in, I believe this error came from loading a saved model (which I did by commit saving, downloading the output, then uploading it back into the kernel). \n\nIll update this if I can reproduce it when I return to training with a saved model.", "votes": null }, { "id": "817177", "postDate": "04/22/2020 23:06:02", "content": "Yes, there is a problem currently with saving/loading in SavedModel format from the TPU to localhost.\nIt works in HD5 format though. The TPU getting started notebook has an example.
", "rawMarkdown": "Yes, there is a problem currently with saving/loading in SavedModel format from the TPU to localhost.\nIt works in HD5 format though. The [TPU getting started notebook](https://www.kaggle.com/mgornergoogle/five-flowers-with-keras-and-xception-on-tpu) has an example.", "votes": null }, { "id": "899263", "postDate": "06/24/2020 05:55:22", "content": "@mgornergoogle While doing KaggleDatasets().get_gcs_path() got below error:\nBackendError: Unexpected response from the service. Response: {'errors': ['Private datasets cannot be copied'], 'error': {'code': 7}, 'wasSuccessful': False}.\nIS there any way to upload in GCS in kaggle?
", "rawMarkdown": "mgornergoogle While doing KaggleDatasets().get_gcs_path() got below error:\nBackendError: Unexpected response from the service. Response: {'errors': ['Private datasets cannot be copied'], 'error': {'code': 7}, 'wasSuccessful': False}.\nIS there any way to upload in GCS in kaggle?", "votes": null }, { "id": "901665", "postDate": "06/25/2020 16:17:48", "content": "Hi @a5bhowmik! I'm looking into this for you, and was hoping you could clarify something for me. When you're getting this error, are you trying to use TPUs with a private dataset?
", "rawMarkdown": "Hi @a5bhowmik! I'm looking into this for you, and was hoping you could clarify something for me. When you're getting this error, are you trying to use TPUs with a private dataset?", "votes": null }, { "id": "904388", "postDate": "06/27/2020 15:30:02", "content": "Yes @jessemostipak . I am trying to use private tfrecords using TPU.
", "rawMarkdown": "Yes @jessemostipak . I am trying to use private tfrecords using TPU.", "votes": null }, { "id": "907320", "postDate": "06/29/2020 21:12:35", "content": "ah, that likely explains it - at this point in time TPUs are not set up to work with private datasets.
", "rawMarkdown": "ah, that likely explains it - at this point in time TPUs are not set up to work with private datasets.", "votes": null }, { "id": "911677", "postDate": "07/01/2020 23:37:40", "content": "@a5bhowmik good news - we've added the ability to work with private datasets and TPUs. you can check out this post for more information.
", "rawMarkdown": "a5bhowmik good news - we've added the ability to work with private datasets and TPUs. you can check out [this post](https://www.kaggle.com/product-feedback/163416) for more information.", "votes": null }, { "id": "970322", "postDate": "08/14/2020 11:05:10", "content": "Hi, Martin.
\nCould you help me. I stack with the same - tf. model cannot save to localhost:
\nif I use save_locally = tf.saved_model.SaveOptions(experimental_io_device='/job:localhost')\nm.save('./model', options=save_locally) # saving in Tensorflow's \"saved model\" format - I get error:
\n\nTypeError Traceback (most recent call last)
\n
\n in
\n 9 # )
\n 10
\n ---> 11 save_locally = tf.saved_model.SaveOptions(experimental_io_device='/job:localhost')
\n 12 m.save('./model', options=save_locally) # saving in Tensorflow's \"saved model\" format
TypeError: init() got an unexpected keyword argument 'experimental_io_device'
\nif I do
\nserved_function = m.call\ntf.saved_model.save(\n m, \n export_dir=\"./model\", \n signatures={'serving_default': served_function}\n)
\nthen - `/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/context.py in sync_executors(self)
\n 656 \"\"\"
\n 657 if self._context_handle:
\n--> 658 pywrap_tfe.TFE_ContextSyncExecutors(self._context_handle)
\n 659 else:
\n 660 raise ValueError(\"Context is not initialized.\")
UnimplementedError: File system scheme '[local]' not implemented (file: './model/variables/variables_temp_968a8eeeea6f43fabe08a7cfcfce1da9/part-00000-of-00001')`
\nThe last code worked yesterday
", "rawMarkdown": "Hi, Martin.\nCould you help me. I stack with the same - tf. model cannot save to localhost:\nif I use `save_locally = tf.saved_model.SaveOptions(experimental_io_device='/job:localhost')\nm.save('./model', options=save_locally) # saving in Tensorflow's \"saved model\" format ` - I get error:\n> TypeError Traceback (most recent call last)\nThe dataset has to be loaded from GCS if you want to use it on TPU.\nUse KaggleDatasets().get_gcs_path() to get a GCS location for the dataset. See kaggle.com/docs/tpu documentation.
", "votes": null, "replies": [ { "id": 815936, "author_name": "rashanarshad", "author_url": "", "post_date": "04/22/2020 00:40:30", "content": "that is actually how im loading the dataset in, I believe this error came from loading a saved model (which I did by commit saving, downloading the output, then uploading it back into the kernel).
\n\nIll update this if I can reproduce it when I return to training with a saved model.
", "votes": null, "replies": [] }, { "id": 817177, "author_name": "mgorner", "author_url": "", "post_date": "04/22/2020 23:06:02", "content": "Yes, there is a problem currently with saving/loading in SavedModel format from the TPU to localhost.\nIt works in HD5 format though. The TPU getting started notebook has an example.
", "votes": null, "replies": [] }, { "id": 970322, "author_name": "vmetenev", "author_url": "", "post_date": "08/14/2020 11:05:10", "content": "Hi, Martin.
\nCould you help me. I stack with the same - tf. model cannot save to localhost:
\nif I use save_locally = tf.saved_model.SaveOptions(experimental_io_device='/job:localhost')\nm.save('./model', options=save_locally) # saving in Tensorflow's \"saved model\" format - I get error:
\n\nTypeError Traceback (most recent call last)
\n
\n in
\n 9 # )
\n 10
\n ---> 11 save_locally = tf.saved_model.SaveOptions(experimental_io_device='/job:localhost')
\n 12 m.save('./model', options=save_locally) # saving in Tensorflow's \"saved model\" format
TypeError: init() got an unexpected keyword argument 'experimental_io_device'
\nif I do
\nserved_function = m.call\ntf.saved_model.save(\n m, \n export_dir=\"./model\", \n signatures={'serving_default': served_function}\n)
\nthen - `/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/context.py in sync_executors(self)
\n 656 \"\"\"
\n 657 if self._context_handle:
\n--> 658 pywrap_tfe.TFE_ContextSyncExecutors(self._context_handle)
\n 659 else:
\n 660 raise ValueError(\"Context is not initialized.\")
UnimplementedError: File system scheme '[local]' not implemented (file: './model/variables/variables_temp_968a8eeeea6f43fabe08a7cfcfce1da9/part-00000-of-00001')`
\nThe last code worked yesterday
", "votes": null, "replies": [] } ] }, { "id": 899263, "author_name": "a5bhowmik", "author_url": "", "post_date": "06/24/2020 05:55:22", "content": "@mgornergoogle While doing KaggleDatasets().get_gcs_path() got below error:\nBackendError: Unexpected response from the service. Response: {'errors': ['Private datasets cannot be copied'], 'error': {'code': 7}, 'wasSuccessful': False}.\nIS there any way to upload in GCS in kaggle?
", "votes": null, "replies": [ { "id": 901665, "author_name": "jessemostipak", "author_url": "", "post_date": "06/25/2020 16:17:48", "content": "Hi @a5bhowmik! I'm looking into this for you, and was hoping you could clarify something for me. When you're getting this error, are you trying to use TPUs with a private dataset?
", "votes": null, "replies": [] }, { "id": 904388, "author_name": "a5bhowmik", "author_url": "", "post_date": "06/27/2020 15:30:02", "content": "Yes @jessemostipak . I am trying to use private tfrecords using TPU.
", "votes": null, "replies": [] }, { "id": 907320, "author_name": "jessemostipak", "author_url": "", "post_date": "06/29/2020 21:12:35", "content": "ah, that likely explains it - at this point in time TPUs are not set up to work with private datasets.
", "votes": null, "replies": [] }, { "id": 911677, "author_name": "jessemostipak", "author_url": "", "post_date": "07/01/2020 23:37:40", "content": "@a5bhowmik good news - we've added the ability to work with private datasets and TPUs. you can check out this post for more information.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "812639": "Error seemingly comes out the blue\n\nI'm trying to iterate on a .take() operation on a dataset I created and am getting this error:\n\n`File system scheme '[local]' not implemented (file: '')`\n\nI can do `data.take(3)` let's say and it returns 3 tensors, but when I do something like:\n\n`for image, label in dataset.take(3):`\n\nI get that error", "814685": "The dataset has to be loaded from GCS if you want to use it on TPU.\nUse KaggleDatasets().get_gcs_path() to get a GCS location for the dataset. See [kaggle.com/docs/tpu](https://www.kaggle.com/docs/tpu) documentation.", "815936": "that is actually how im loading the dataset in, I believe this error came from loading a saved model (which I did by commit saving, downloading the output, then uploading it back into the kernel). \n\nIll update this if I can reproduce it when I return to training with a saved model.", "817177": "Yes, there is a problem currently with saving/loading in SavedModel format from the TPU to localhost.\nIt works in HD5 format though. The [TPU getting started notebook](https://www.kaggle.com/mgornergoogle/five-flowers-with-keras-and-xception-on-tpu) has an example.", "899263": "mgornergoogle While doing KaggleDatasets().get_gcs_path() got below error:\nBackendError: Unexpected response from the service. Response: {'errors': ['Private datasets cannot be copied'], 'error': {'code': 7}, 'wasSuccessful': False}.\nIS there any way to upload in GCS in kaggle?", "901665": "Hi @a5bhowmik! I'm looking into this for you, and was hoping you could clarify something for me. When you're getting this error, are you trying to use TPUs with a private dataset?", "904388": "Yes @jessemostipak . I am trying to use private tfrecords using TPU.", "907320": "ah, that likely explains it - at this point in time TPUs are not set up to work with private datasets.", "911677": "a5bhowmik good news - we've added the ability to work with private datasets and TPUs. you can check out [this post](https://www.kaggle.com/product-feedback/163416) for more information.", "970322": "Hi, Martin.\nCould you help me. I stack with the same - tf. model cannot save to localhost:\nif I use `save_locally = tf.saved_model.SaveOptions(experimental_io_device='/job:localhost')\nm.save('./model', options=save_locally) # saving in Tensorflow's \"saved model\" format ` - I get error:\n> TypeError Traceback (most recent call last)\n