{
  "id": 402516,
  "title": "runtime memory (RAM) of ensemble model = one base model. is it possible in tflite?",
  "url": "/competitions/asl-signs/discussion/402516",
  "author_name": "hengck23",
  "post_date": "2023-04-18T17:40:51.935000",
  "votes": 1,
  "comment_count": 0,
  "views": 0,
  "content": "<p>is it possible to reuse the memory so that the runtime memory (RAM) of ensemble model (i.e. N x similar base models) is the same as RAM of one  base model?</p>\n<pre><code>    class TFModel(tf.Module):\n        def __init__(self):\n            super(TFModel, self).__init__()\n            self.input_net  = tf.saved_model.load(input_net_k_tf_file)\n            self.single_net = [\n                tf.saved_model.load(single_net0_p_tf_file),\n                tf.saved_model.load(single_net1_p_tf_file),\n                tf.saved_model.load(single_net2_p_tf_file),\n                tf.saved_model.load(single_net3_p_tf_file),\n            ]\n\n            self.input_net.trainable = False\n            for s in self.single_net:\n                s.trainable = False\n\n\n        @tf.function(input_signature=[\n            tf.TensorSpec(shape=[None, 543, 3], dtype=tf.float32, name='inputs')\n        ])\n        def __call__(self, inputs):\n\n            x  = self.input_net(inputs)\n\n            y  = 0\n            S = len(self.single_net)\n            for s in range(S):\n                y += self.single_net[s](inputs=x)['outputs']   \n            outputs = y/S\n            return {'outputs': outputs }\n\n\n# self.single_net  is a list of smiliar base models\n</code></pre>",
  "messages": [
    {
      "id": 2226160,
      "postDate": "2023-04-18T17:40:51.937Z",
      "content": "<p>is it possible to reuse the memory so that the runtime memory (RAM) of ensemble model (i.e. N x similar base models) is the same as RAM of one  base model?</p>\n<pre><code>    class TFModel(tf.Module):\n        def __init__(self):\n            super(TFModel, self).__init__()\n            self.input_net  = tf.saved_model.load(input_net_k_tf_file)\n            self.single_net = [\n                tf.saved_model.load(single_net0_p_tf_file),\n                tf.saved_model.load(single_net1_p_tf_file),\n                tf.saved_model.load(single_net2_p_tf_file),\n                tf.saved_model.load(single_net3_p_tf_file),\n            ]\n\n            self.input_net.trainable = False\n            for s in self.single_net:\n                s.trainable = False\n\n\n        @tf.function(input_signature=[\n            tf.TensorSpec(shape=[None, 543, 3], dtype=tf.float32, name='inputs')\n        ])\n        def __call__(self, inputs):\n\n            x  = self.input_net(inputs)\n\n            y  = 0\n            S = len(self.single_net)\n            for s in range(S):\n                y += self.single_net[s](inputs=x)['outputs']   \n            outputs = y/S\n            return {'outputs': outputs }\n\n\n# self.single_net  is a list of smiliar base models\n</code></pre>",
      "rawMarkdown": "is it possible to reuse the memory so that the runtime memory (RAM) of ensemble model (i.e. N x similar base models) is the same as RAM of one  base model?\n\n```\n\n\tclass TFModel(tf.Module):\n\t\tdef __init__(self):\n\t\t\tsuper(TFModel, self).__init__()\n\t\t\tself.input_net  = tf.saved_model.load(input_net_k_tf_file)\n\t\t\tself.single_net = [\n\t\t\t\ttf.saved_model.load(single_net0_p_tf_file),\n\t\t\t\ttf.saved_model.load(single_net1_p_tf_file),\n\t\t\t\ttf.saved_model.load(single_net2_p_tf_file),\n\t\t\t\ttf.saved_model.load(single_net3_p_tf_file),\n\t\t\t]\n\n\t\t\tself.input_net.trainable = False\n\t\t\tfor s in self.single_net:\n\t\t\t\ts.trainable = False\n\n\n\t\t@tf.function(input_signature=[\n\t\t\ttf.TensorSpec(shape=[None, 543, 3], dtype=tf.float32, name='inputs')\n\t\t])\n\t\tdef __call__(self, inputs):\n\t\t\t  \n\t\t\tx  = self.input_net(inputs)\n\n\t\t\ty  = 0\n\t\t\tS = len(self.single_net)\n\t\t\tfor s in range(S):\n\t\t\t\ty += self.single_net[s](inputs=x)['outputs']   \n\t\t\toutputs = y/S\n\t\t\treturn {'outputs': outputs }\n\n\n# self.single_net  is a list of smiliar base models\n```",
      "votes": 1
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2226160": "is it possible to reuse the memory so that the runtime memory (RAM) of ensemble model (i.e. N x similar base models) is the same as RAM of one  base model?\n\n```\n\n\tclass TFModel(tf.Module):\n\t\tdef __init__(self):\n\t\t\tsuper(TFModel, self).__init__()\n\t\t\tself.input_net  = tf.saved_model.load(input_net_k_tf_file)\n\t\t\tself.single_net = [\n\t\t\t\ttf.saved_model.load(single_net0_p_tf_file),\n\t\t\t\ttf.saved_model.load(single_net1_p_tf_file),\n\t\t\t\ttf.saved_model.load(single_net2_p_tf_file),\n\t\t\t\ttf.saved_model.load(single_net3_p_tf_file),\n\t\t\t]\n\n\t\t\tself.input_net.trainable = False\n\t\t\tfor s in self.single_net:\n\t\t\t\ts.trainable = False\n\n\n\t\t@tf.function(input_signature=[\n\t\t\ttf.TensorSpec(shape=[None, 543, 3], dtype=tf.float32, name='inputs')\n\t\t])\n\t\tdef __call__(self, inputs):\n\t\t\t  \n\t\t\tx  = self.input_net(inputs)\n\n\t\t\ty  = 0\n\t\t\tS = len(self.single_net)\n\t\t\tfor s in range(S):\n\t\t\t\ty += self.single_net[s](inputs=x)['outputs']   \n\t\t\toutputs = y/S\n\t\t\treturn {'outputs': outputs }\n\n\n# self.single_net  is a list of smiliar base models\n```"
  }
}