{
  "id": 166149,
  "title": "Re-saving baseline model",
  "url": "/competitions/landmark-retrieval-2020/discussion/166149",
  "author_name": "",
  "post_date": "2020-07-12T01:01:51.188519800Z",
  "votes": 13,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Let's say we want to use the baseline model as part of more complex pipeline. In order to do that, we would do something like:</p>\n\n<p>```\nclass ComplexModel(tf.Module):</p>\n\n<pre><code>@tf.function(input_signature=[\n  tf.TensorSpec(shape=[None, None, 3], dtype=tf.uint8, name='x')\n])\ndef __call__(self, input_image):\n    x = do_some_work_with(input_image)\n    y = do_more_work_with(model, x, input_image)\n    return y\n</code></pre>\n\n<p>```</p>\n\n<p>Where <code>model</code> is a baseline model provided, e.g. loaded like that:</p>\n\n<p><code>\nmodel = tf.saved_model.load(\n  '/kaggle/input/baseline-landmark-retrieval-model/baseline_landmark_retrieval_model'\n).signatures['serving_default']\n</code></p>\n\n<p>Now, it turns out something like that works fine:</p>\n\n<p><code>\ncomplex_model = ComplexModel()\nembedding = complex_model(image_tensor)\n</code></p>\n\n<p>But, if we further do:</p>\n\n<p><code>\ntf.saved_model.save(complex_model, 'complex_model')\nimported = tf.saved_model.load('complex_model')\nembedding = imported(image_tensor)\n</code></p>\n\n<p>Some strange errors are emitted, like:</p>\n\n<p>```\nFailedPreconditionError:  Attempting to use uninitialized value StatefulPartitionedCall/StatefulPartitionedCall_1/resnet_v1_101/block3/unit_5/bottleneck_v1/conv2/BatchNorm/beta\n     [[{{node StatefulPartitionedCall/StatefulPartitionedCall_1/resnet_v1_101/block3/unit_5/bottleneck_v1/conv2/BatchNorm/beta/read}}]] [Op:__inference_restored_function_body_27759]</p>\n\n<p>Function call stack:\nrestored_function_body\n```</p>\n\n<p>It seems like Tensorflow is unable to save variables from the baseline model. When I look at size of saved model files (in the example, in <code>complex_model</code> checkpoint), the size is much less than ~180 MB originally taken by baseline model, it seems like no baseline model's variable is saved - the files in <code>complex_model/variables</code> take kilobytes, not megabytes...</p>\n\n<p>Is there any way not to lose variable information after re-saving baseline model (and preferably, saving a <code>tf.Module</code> that uses baseline model inside) using <code>tf.saved_model.save()</code>?</p>",
  "messages": [
    {
      "id": "925275",
      "postDate": "07/12/2020 01:01:51",
      "content": "<p>Let's say we want to use the baseline model as part of more complex pipeline. In order to do that, we would do something like:</p>\n\n<p>```\nclass ComplexModel(tf.Module):</p>\n\n<pre><code>@tf.function(input_signature=[\n  tf.TensorSpec(shape=[None, None, 3], dtype=tf.uint8, name='x')\n])\ndef __call__(self, input_image):\n    x = do_some_work_with(input_image)\n    y = do_more_work_with(model, x, input_image)\n    return y\n</code></pre>\n\n<p>```</p>\n\n<p>Where <code>model</code> is a baseline model provided, e.g. loaded like that:</p>\n\n<p><code>\nmodel = tf.saved_model.load(\n  '/kaggle/input/baseline-landmark-retrieval-model/baseline_landmark_retrieval_model'\n).signatures['serving_default']\n</code></p>\n\n<p>Now, it turns out something like that works fine:</p>\n\n<p><code>\ncomplex_model = ComplexModel()\nembedding = complex_model(image_tensor)\n</code></p>\n\n<p>But, if we further do:</p>\n\n<p><code>\ntf.saved_model.save(complex_model, 'complex_model')\nimported = tf.saved_model.load('complex_model')\nembedding = imported(image_tensor)\n</code></p>\n\n<p>Some strange errors are emitted, like:</p>\n\n<p>```\nFailedPreconditionError:  Attempting to use uninitialized value StatefulPartitionedCall/StatefulPartitionedCall_1/resnet_v1_101/block3/unit_5/bottleneck_v1/conv2/BatchNorm/beta\n     [[{{node StatefulPartitionedCall/StatefulPartitionedCall_1/resnet_v1_101/block3/unit_5/bottleneck_v1/conv2/BatchNorm/beta/read}}]] [Op:__inference_restored_function_body_27759]</p>\n\n<p>Function call stack:\nrestored_function_body\n```</p>\n\n<p>It seems like Tensorflow is unable to save variables from the baseline model. When I look at size of saved model files (in the example, in <code>complex_model</code> checkpoint), the size is much less than ~180 MB originally taken by baseline model, it seems like no baseline model's variable is saved - the files in <code>complex_model/variables</code> take kilobytes, not megabytes...</p>\n\n<p>Is there any way not to lose variable information after re-saving baseline model (and preferably, saving a <code>tf.Module</code> that uses baseline model inside) using <code>tf.saved_model.save()</code>?</p>",
      "rawMarkdown": "Let's say we want to use the baseline model as part of more complex pipeline. In order to do that, we would do something like:\n\n```\nclass ComplexModel(tf.Module):\n\n    @tf.function(input_signature=[\n      tf.TensorSpec(shape=[None, None, 3], dtype=tf.uint8, name='x')\n    ])\n    def __call__(self, input_image):\n        x = do_some_work_with(input_image)\n        y = do_more_work_with(model, x, input_image)\n        return y\n```\n\nWhere `model` is a baseline model provided, e.g. loaded like that:\n\n```\nmodel = tf.saved_model.load(\n  '/kaggle/input/baseline-landmark-retrieval-model/baseline_landmark_retrieval_model'\n).signatures['serving_default']\n```\n\nNow, it turns out something like that works fine:\n\n```\ncomplex_model = ComplexModel()\nembedding = complex_model(image_tensor)\n```\n\nBut, if we further do:\n\n```\ntf.saved_model.save(complex_model, 'complex_model')\nimported = tf.saved_model.load('complex_model')\nembedding = imported(image_tensor)\n```\n\nSome strange errors are emitted, like:\n\n```\nFailedPreconditionError:  Attempting to use uninitialized value StatefulPartitionedCall/StatefulPartitionedCall_1/resnet_v1_101/block3/unit_5/bottleneck_v1/conv2/BatchNorm/beta\n\t [[{{node StatefulPartitionedCall/StatefulPartitionedCall_1/resnet_v1_101/block3/unit_5/bottleneck_v1/conv2/BatchNorm/beta/read}}]] [Op:__inference_restored_function_body_27759]\n\nFunction call stack:\nrestored_function_body\n```\n\nIt seems like Tensorflow is unable to save variables from the baseline model. When I look at size of saved model files (in the example, in `complex_model` checkpoint), the size is much less than ~180 MB originally taken by baseline model, it seems like no baseline model's variable is saved - the files in `complex_model/variables` take kilobytes, not megabytes...\n\nIs there any way not to lose variable information after re-saving baseline model (and preferably, saving a `tf.Module` that uses baseline model inside) using `tf.saved_model.save()`?",
      "votes": null
    },
    {
      "id": "925319",
      "postDate": "07/12/2020 02:35:29",
      "content": "<p>you can manually save them and load them. Please have a look at this page near the bottom. I guess you will get your answer. \n<a href=\"https://www.tensorflow.org/tutorials/keras/save_and_load\">https://www.tensorflow.org/tutorials/keras/save_and_load</a></p>",
      "rawMarkdown": "you can manually save them and load them. Please have a look at this page near the bottom. I guess you will get your answer. \nhttps://www.tensorflow.org/tutorials/keras/save_and_load",
      "votes": null
    },
    {
      "id": "925615",
      "postDate": "07/12/2020 07:29:17",
      "content": "<p>I'm trying the same thing, but resaved model has some problems. I want to know the proper way.</p>",
      "rawMarkdown": "I'm trying the same thing, but resaved model has some problems. I want to know the proper way.",
      "votes": null
    },
    {
      "id": "939158",
      "postDate": "07/22/2020 04:14:07",
      "content": "<p>Any luck with this?</p>",
      "rawMarkdown": "Any luck with this?",
      "votes": null
    },
    {
      "id": "959641",
      "postDate": "08/05/2020 19:11:56",
      "content": "<p>It doesn't work whether you load the baseline by <code>tf.saved_model.load()</code> or by <code>tf.keras.models.load_model()</code>. When printing the <code>model.variables</code> or <code>model.trainable_variables</code>, you get empty list. I don't know whether there is any way to fine-tune the baseline model; it seems the baseline that the organizer provided refuses to disclose any weight information and it is probably impossible to do any stuff on top of this baseline other than inference.</p>",
      "rawMarkdown": "It doesn't work whether you load the baseline by `tf.saved_model.load()` or by `tf.keras.models.load_model()`. When printing the `model.variables` or `model.trainable_variables`, you get empty list. I don't know whether there is any way to fine-tune the baseline model; it seems the baseline that the organizer provided refuses to disclose any weight information and it is probably impossible to do any stuff on top of this baseline other than inference.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 925319,
      "author_name": "redwankarimsony",
      "author_url": "",
      "post_date": "07/12/2020 02:35:29",
      "content": "<p>you can manually save them and load them. Please have a look at this page near the bottom. I guess you will get your answer. \n<a href=\"https://www.tensorflow.org/tutorials/keras/save_and_load\">https://www.tensorflow.org/tutorials/keras/save_and_load</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 959641,
          "author_name": "xirenzhou",
          "author_url": "",
          "post_date": "08/05/2020 19:11:56",
          "content": "<p>It doesn't work whether you load the baseline by <code>tf.saved_model.load()</code> or by <code>tf.keras.models.load_model()</code>. When printing the <code>model.variables</code> or <code>model.trainable_variables</code>, you get empty list. I don't know whether there is any way to fine-tune the baseline model; it seems the baseline that the organizer provided refuses to disclose any weight information and it is probably impossible to do any stuff on top of this baseline other than inference.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 925615,
      "author_name": "toshik",
      "author_url": "",
      "post_date": "07/12/2020 07:29:17",
      "content": "<p>I'm trying the same thing, but resaved model has some problems. I want to know the proper way.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 939158,
      "author_name": "chandanverma",
      "author_url": "",
      "post_date": "07/22/2020 04:14:07",
      "content": "<p>Any luck with this?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "925275": "Let's say we want to use the baseline model as part of more complex pipeline. In order to do that, we would do something like:\n\n```\nclass ComplexModel(tf.Module):\n\n    @tf.function(input_signature=[\n      tf.TensorSpec(shape=[None, None, 3], dtype=tf.uint8, name='x')\n    ])\n    def __call__(self, input_image):\n        x = do_some_work_with(input_image)\n        y = do_more_work_with(model, x, input_image)\n        return y\n```\n\nWhere `model` is a baseline model provided, e.g. loaded like that:\n\n```\nmodel = tf.saved_model.load(\n  '/kaggle/input/baseline-landmark-retrieval-model/baseline_landmark_retrieval_model'\n).signatures['serving_default']\n```\n\nNow, it turns out something like that works fine:\n\n```\ncomplex_model = ComplexModel()\nembedding = complex_model(image_tensor)\n```\n\nBut, if we further do:\n\n```\ntf.saved_model.save(complex_model, 'complex_model')\nimported = tf.saved_model.load('complex_model')\nembedding = imported(image_tensor)\n```\n\nSome strange errors are emitted, like:\n\n```\nFailedPreconditionError:  Attempting to use uninitialized value StatefulPartitionedCall/StatefulPartitionedCall_1/resnet_v1_101/block3/unit_5/bottleneck_v1/conv2/BatchNorm/beta\n\t [[{{node StatefulPartitionedCall/StatefulPartitionedCall_1/resnet_v1_101/block3/unit_5/bottleneck_v1/conv2/BatchNorm/beta/read}}]] [Op:__inference_restored_function_body_27759]\n\nFunction call stack:\nrestored_function_body\n```\n\nIt seems like Tensorflow is unable to save variables from the baseline model. When I look at size of saved model files (in the example, in `complex_model` checkpoint), the size is much less than ~180 MB originally taken by baseline model, it seems like no baseline model's variable is saved - the files in `complex_model/variables` take kilobytes, not megabytes...\n\nIs there any way not to lose variable information after re-saving baseline model (and preferably, saving a `tf.Module` that uses baseline model inside) using `tf.saved_model.save()`?",
    "925319": "you can manually save them and load them. Please have a look at this page near the bottom. I guess you will get your answer. \nhttps://www.tensorflow.org/tutorials/keras/save_and_load",
    "925615": "I'm trying the same thing, but resaved model has some problems. I want to know the proper way.",
    "939158": "Any luck with this?",
    "959641": "It doesn't work whether you load the baseline by `tf.saved_model.load()` or by `tf.keras.models.load_model()`. When printing the `model.variables` or `model.trainable_variables`, you get empty list. I don't know whether there is any way to fine-tune the baseline model; it seems the baseline that the organizer provided refuses to disclose any weight information and it is probably impossible to do any stuff on top of this baseline other than inference."
  },
  "source": "meta"
}