{
  "id": 396973,
  "title": "Padding during inference ",
  "url": "/competitions/asl-signs/discussion/396973",
  "author_name": "",
  "post_date": "2023-03-23T15:33:12.553236Z",
  "votes": null,
  "comment_count": 6,
  "views": 0,
  "content": "<p>Hi all, </p>\n<p>During training I am padding inputs to (None, max_frames, flattened_landmarks) using tf.keras.pad_sequences and am having trouble doing the same when creating the inference model. </p>\n<p>def get_inference_model(model):<br>\n    inputs = tf.keras.Input((543, 3), dtype=tf.float32, name=\"inputs\")<br>\n    x = tf.where(tf.math.is_nan(x), tf.zeros_like(x), x)<br>\n    x = tf.concat([x[…,i] for i in range(3)], -1) # flatten <br>\n    x = tf.expand_dims(x,axis = 0)    </p>\n<pre><code> *Shape here is (None, 1, 1629) which is incompatible with the padded training inputs to my model (None, max_frames 1629)*\n\nx = model(x)    \noutput = tf.keras.layers.Activation(activation=\"linear\", name=\"outputs\")(x)\ninference_model = tf.keras.Model(inputs=inputs, outputs=output) \ninference_model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(), metrics=[\"accuracy\"])\nreturn inference_model***\n</code></pre>\n<p>Any help would be appreciated! </p>",
  "messages": [
    {
      "id": "2193912",
      "postDate": "03/23/2023 15:33:12",
      "content": "<p>Hi all, </p>\n<p>During training I am padding inputs to (None, max_frames, flattened_landmarks) using tf.keras.pad_sequences and am having trouble doing the same when creating the inference model. </p>\n<p>def get_inference_model(model):<br>\n    inputs = tf.keras.Input((543, 3), dtype=tf.float32, name=\"inputs\")<br>\n    x = tf.where(tf.math.is_nan(x), tf.zeros_like(x), x)<br>\n    x = tf.concat([x[…,i] for i in range(3)], -1) # flatten <br>\n    x = tf.expand_dims(x,axis = 0)    </p>\n<pre><code> *Shape here is (None, 1, 1629) which is incompatible with the padded training inputs to my model (None, max_frames 1629)*\n\nx = model(x)    \noutput = tf.keras.layers.Activation(activation=\"linear\", name=\"outputs\")(x)\ninference_model = tf.keras.Model(inputs=inputs, outputs=output) \ninference_model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(), metrics=[\"accuracy\"])\nreturn inference_model***\n</code></pre>\n<p>Any help would be appreciated! </p>",
      "rawMarkdown": "Hi all, \n\nDuring training I am padding inputs to (None, max_frames, flattened_landmarks) using tf.keras.pad_sequences and am having trouble doing the same when creating the inference model. \n\ndef get_inference_model(model):\n    inputs = tf.keras.Input((543, 3), dtype=tf.float32, name=\"inputs\")\n    x = tf.where(tf.math.is_nan(x), tf.zeros_like(x), x)\n    x = tf.concat([x[...,i] for i in range(3)], -1) # flatten \n    x = tf.expand_dims(x,axis = 0)    \n\n     *Shape here is (None, 1, 1629) which is incompatible with the padded training inputs to my model (None, max_frames 1629)*\n\n    x = model(x)    \n    output = tf.keras.layers.Activation(activation=\"linear\", name=\"outputs\")(x)\n    inference_model = tf.keras.Model(inputs=inputs, outputs=output) \n    inference_model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(), metrics=[\"accuracy\"])\n    return inference_model***\n\nAny help would be appreciated!",
      "votes": null
    },
    {
      "id": "2196648",
      "postDate": "03/25/2023 15:02:56",
      "content": "<p>Use tf.pad function instead </p>",
      "rawMarkdown": "Use tf.pad function instead",
      "votes": null
    },
    {
      "id": "2197048",
      "postDate": "03/25/2023 20:34:59",
      "content": "<blockquote>\n  <p>Use tf.pad function instead</p>\n</blockquote>\n<p>tf.pad still requires me to know the number of frames which is variable during inference</p>",
      "rawMarkdown": "> Use tf.pad function instead\n\ntf.pad still requires me to know the number of frames which is variable during inference",
      "votes": null
    },
    {
      "id": "2197565",
      "postDate": "03/26/2023 08:25:49",
      "content": "<p>during inference (for submission), batch size is one.<br>\nhence we have:<br>\nbatch size = 1 (known)<br>\nlength= none (unknown)</p>\n<p>you have to rewrite your tf code for this.</p>\n<p>train and inference tf code need not to be exactly the same</p>",
      "rawMarkdown": "during inference (for submission), batch size is one.\nhence we have:\nbatch size = 1 (known)\nlength= none (unknown)\n\nyou have to rewrite your tf code for this.\n\ntrain and inference tf code need not to be exactly the same",
      "votes": null
    },
    {
      "id": "2198039",
      "postDate": "03/26/2023 16:32:14",
      "content": "<p>My model expects the videos to be padded to a certain length so I am trying to do that at inference </p>",
      "rawMarkdown": "My model expects the videos to be padded to a certain length so I am trying to do that at inference",
      "votes": null
    },
    {
      "id": "2198076",
      "postDate": "03/26/2023 16:59:42",
      "content": "<p>Do you consider using tf.shape to get unknown shape values?</p>",
      "rawMarkdown": "Do you consider using tf.shape to get unknown shape values?",
      "votes": null
    },
    {
      "id": "2198345",
      "postDate": "03/26/2023 23:35:42",
      "content": "<p>tf.image.resize_with_pad(x, max_length, len(point_landmarks)) ended up being the solution I was looking for </p>",
      "rawMarkdown": "tf.image.resize_with_pad(x, max_length, len(point_landmarks)) ended up being the solution I was looking for",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2196648,
      "author_name": "georgemegre",
      "author_url": "",
      "post_date": "03/25/2023 15:02:56",
      "content": "<p>Use tf.pad function instead </p>",
      "votes": null,
      "replies": [
        {
          "id": 2197048,
          "author_name": "researchlad",
          "author_url": "",
          "post_date": "03/25/2023 20:34:59",
          "content": "<blockquote>\n  <p>Use tf.pad function instead</p>\n</blockquote>\n<p>tf.pad still requires me to know the number of frames which is variable during inference</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2197565,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/26/2023 08:25:49",
      "content": "<p>during inference (for submission), batch size is one.<br>\nhence we have:<br>\nbatch size = 1 (known)<br>\nlength= none (unknown)</p>\n<p>you have to rewrite your tf code for this.</p>\n<p>train and inference tf code need not to be exactly the same</p>",
      "votes": null,
      "replies": [
        {
          "id": 2198039,
          "author_name": "researchlad",
          "author_url": "",
          "post_date": "03/26/2023 16:32:14",
          "content": "<p>My model expects the videos to be padded to a certain length so I am trying to do that at inference </p>",
          "votes": null,
          "replies": [
            {
              "id": 2198076,
              "author_name": "meowmeowmeowmeowmeow",
              "author_url": "",
              "post_date": "03/26/2023 16:59:42",
              "content": "<p>Do you consider using tf.shape to get unknown shape values?</p>",
              "votes": null,
              "replies": []
            }
          ]
        },
        {
          "id": 2198345,
          "author_name": "researchlad",
          "author_url": "",
          "post_date": "03/26/2023 23:35:42",
          "content": "<p>tf.image.resize_with_pad(x, max_length, len(point_landmarks)) ended up being the solution I was looking for </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2193912": "Hi all, \n\nDuring training I am padding inputs to (None, max_frames, flattened_landmarks) using tf.keras.pad_sequences and am having trouble doing the same when creating the inference model. \n\ndef get_inference_model(model):\n    inputs = tf.keras.Input((543, 3), dtype=tf.float32, name=\"inputs\")\n    x = tf.where(tf.math.is_nan(x), tf.zeros_like(x), x)\n    x = tf.concat([x[...,i] for i in range(3)], -1) # flatten \n    x = tf.expand_dims(x,axis = 0)    \n\n     *Shape here is (None, 1, 1629) which is incompatible with the padded training inputs to my model (None, max_frames 1629)*\n\n    x = model(x)    \n    output = tf.keras.layers.Activation(activation=\"linear\", name=\"outputs\")(x)\n    inference_model = tf.keras.Model(inputs=inputs, outputs=output) \n    inference_model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(), metrics=[\"accuracy\"])\n    return inference_model***\n\nAny help would be appreciated!",
    "2196648": "Use tf.pad function instead",
    "2197048": "> Use tf.pad function instead\n\ntf.pad still requires me to know the number of frames which is variable during inference",
    "2197565": "during inference (for submission), batch size is one.\nhence we have:\nbatch size = 1 (known)\nlength= none (unknown)\n\nyou have to rewrite your tf code for this.\n\ntrain and inference tf code need not to be exactly the same",
    "2198039": "My model expects the videos to be padded to a certain length so I am trying to do that at inference",
    "2198076": "Do you consider using tf.shape to get unknown shape values?",
    "2198345": "tf.image.resize_with_pad(x, max_length, len(point_landmarks)) ended up being the solution I was looking for"
  },
  "source": "meta"
}