{
  "id": 207280,
  "title": "tf.function with input_signature make my prediction x3 ~ x5 faster.",
  "url": "/competitions/riiid-test-answer-prediction/discussion/207280",
  "author_name": "nadare",
  "post_date": "2020-12-29T01:01:50.101000",
  "votes": 7,
  "comment_count": 0,
  "views": 0,
  "content": "<p>example</p>\n<pre><code>    @tf.function(input_signature=[\n        tf.TensorSpec(shape=None, dtype=tf.int64),\n        tf.TensorSpec(shape=None, dtype=tf.int32),\n        tf.TensorSpec(shape=[], dtype=tf.bool),\n    ]\n    )\n    def call(self, user_id, content_id, training=False):\n        N = len(user_ix)\n        user_id = tf.reshape(user_id, (-1,))\n        content_id = tf.reshape(content_id, (-1,))\n</code></pre>\n<ol>\n<li>Use <a href=\"https://www.kaggle.com/tf.function\" target=\"_blank\">@tf.function</a> with input_signature</li>\n<li>Use len() for batch_size in your function</li>\n<li>Specify tensor shape with tf.reshape</li>\n</ol>\n<p>result</p>\n<p>before</p>\n<pre><code>loop0 preprocess: 0.2186906337738037 sec\nloop0 prediction: 1.830209493637085 sec\nloop1 preprocess: 0.17534494400024414 sec\nloop1 prediction: 0.24773740768432617 sec\nloop2 preprocess: 0.17480683326721191 sec\nloop2 prediction: 0.23581433296203613 sec\nloop3 preprocess: 0.13235163688659668 sec\nloop3 prediction: 0.23711228370666504 sec\n</code></pre>\n<p>after</p>\n<pre><code>loop0 preprocess: 9.557458639144897 sec\nloop0 prediction: 15.090033054351807 sec\nloop1 preprocess: 0.03805422782897949 sec\nloop1 prediction: 0.08428549766540527 sec\nloop2 preprocess: 0.03657054901123047 sec\nloop2 prediction: 0.08363556861877441 sec\nloop3 preprocess: 0.04673504829406738 sec\nloop3 prediction: 0.08835339546203613 sec\n</code></pre>",
  "messages": [
    {
      "id": 1130364,
      "postDate": "2020-12-29T01:01:50.103Z",
      "content": "<p>example</p>\n<pre><code>    @tf.function(input_signature=[\n        tf.TensorSpec(shape=None, dtype=tf.int64),\n        tf.TensorSpec(shape=None, dtype=tf.int32),\n        tf.TensorSpec(shape=[], dtype=tf.bool),\n    ]\n    )\n    def call(self, user_id, content_id, training=False):\n        N = len(user_ix)\n        user_id = tf.reshape(user_id, (-1,))\n        content_id = tf.reshape(content_id, (-1,))\n</code></pre>\n<ol>\n<li>Use <a href=\"https://www.kaggle.com/tf.function\" target=\"_blank\">@tf.function</a> with input_signature</li>\n<li>Use len() for batch_size in your function</li>\n<li>Specify tensor shape with tf.reshape</li>\n</ol>\n<p>result</p>\n<p>before</p>\n<pre><code>loop0 preprocess: 0.2186906337738037 sec\nloop0 prediction: 1.830209493637085 sec\nloop1 preprocess: 0.17534494400024414 sec\nloop1 prediction: 0.24773740768432617 sec\nloop2 preprocess: 0.17480683326721191 sec\nloop2 prediction: 0.23581433296203613 sec\nloop3 preprocess: 0.13235163688659668 sec\nloop3 prediction: 0.23711228370666504 sec\n</code></pre>\n<p>after</p>\n<pre><code>loop0 preprocess: 9.557458639144897 sec\nloop0 prediction: 15.090033054351807 sec\nloop1 preprocess: 0.03805422782897949 sec\nloop1 prediction: 0.08428549766540527 sec\nloop2 preprocess: 0.03657054901123047 sec\nloop2 prediction: 0.08363556861877441 sec\nloop3 preprocess: 0.04673504829406738 sec\nloop3 prediction: 0.08835339546203613 sec\n</code></pre>",
      "rawMarkdown": "example\n```\n    @tf.function(input_signature=[\n        tf.TensorSpec(shape=None, dtype=tf.int64),\n        tf.TensorSpec(shape=None, dtype=tf.int32),\n        tf.TensorSpec(shape=[], dtype=tf.bool),\n    ]\n    )\n    def call(self, user_id, content_id, training=False):\n        N = len(user_ix)\n        user_id = tf.reshape(user_id, (-1,))\n        content_id = tf.reshape(content_id, (-1,))\n```\n\n1. Use @tf.function with input_signature\n2. Use len() for batch_size in your function\n3. Specify tensor shape with tf.reshape\n\nresult\n\nbefore\n```\nloop0 preprocess: 0.2186906337738037 sec\nloop0 prediction: 1.830209493637085 sec\nloop1 preprocess: 0.17534494400024414 sec\nloop1 prediction: 0.24773740768432617 sec\nloop2 preprocess: 0.17480683326721191 sec\nloop2 prediction: 0.23581433296203613 sec\nloop3 preprocess: 0.13235163688659668 sec\nloop3 prediction: 0.23711228370666504 sec\n```\n\nafter\n```\nloop0 preprocess: 9.557458639144897 sec\nloop0 prediction: 15.090033054351807 sec\nloop1 preprocess: 0.03805422782897949 sec\nloop1 prediction: 0.08428549766540527 sec\nloop2 preprocess: 0.03657054901123047 sec\nloop2 prediction: 0.08363556861877441 sec\nloop3 preprocess: 0.04673504829406738 sec\nloop3 prediction: 0.08835339546203613 sec\n```",
      "votes": 7
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1130364": "example\n```\n    @tf.function(input_signature=[\n        tf.TensorSpec(shape=None, dtype=tf.int64),\n        tf.TensorSpec(shape=None, dtype=tf.int32),\n        tf.TensorSpec(shape=[], dtype=tf.bool),\n    ]\n    )\n    def call(self, user_id, content_id, training=False):\n        N = len(user_ix)\n        user_id = tf.reshape(user_id, (-1,))\n        content_id = tf.reshape(content_id, (-1,))\n```\n\n1. Use @tf.function with input_signature\n2. Use len() for batch_size in your function\n3. Specify tensor shape with tf.reshape\n\nresult\n\nbefore\n```\nloop0 preprocess: 0.2186906337738037 sec\nloop0 prediction: 1.830209493637085 sec\nloop1 preprocess: 0.17534494400024414 sec\nloop1 prediction: 0.24773740768432617 sec\nloop2 preprocess: 0.17480683326721191 sec\nloop2 prediction: 0.23581433296203613 sec\nloop3 preprocess: 0.13235163688659668 sec\nloop3 prediction: 0.23711228370666504 sec\n```\n\nafter\n```\nloop0 preprocess: 9.557458639144897 sec\nloop0 prediction: 15.090033054351807 sec\nloop1 preprocess: 0.03805422782897949 sec\nloop1 prediction: 0.08428549766540527 sec\nloop2 preprocess: 0.03657054901123047 sec\nloop2 prediction: 0.08363556861877441 sec\nloop3 preprocess: 0.04673504829406738 sec\nloop3 prediction: 0.08835339546203613 sec\n```"
  }
}