{
  "id": 390651,
  "title": "Dealing with variable sequence lengths in combination with TfLite requirement",
  "url": "/competitions/asl-signs/discussion/390651",
  "author_name": "Wondering Alice",
  "post_date": "2023-02-26T14:55:45.944000",
  "votes": 6,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi all,</p>\n<p>The data we have (and the data the model should be able to use) are variable length sequences. <br>\nAs far as I see, and within the TfLite constraints, there are two possible approaches to this:</p>\n<p>1) Fix the sequence length during training (with truncating or padding as a preprocessing step).<br>\n-&gt; since sequences at inference time can be either longer or shorter that the fixed length at training time, this needs to be taken care of in the inference model as well<br>\n-&gt; Does anyone know how to do this??</p>\n<p>2) Train a truly variable sequence length model <br>\n-&gt; Does this work with the TfLite inference flow?<br>\n-&gt; If yes: has anyone figured out how to write a (parquet) Data loader to deal with variable length batches?</p>\n<p>Since these are technical issues that are very hard to overcome for TfLite newbies, it would be much appreciated if anyone (and especially the Google Tf or TfLite Gurus) could help here!</p>",
  "messages": [
    {
      "id": 2160306,
      "postDate": "2023-02-26T14:55:45.943Z",
      "content": "<p>Hi all,</p>\n<p>The data we have (and the data the model should be able to use) are variable length sequences. <br>\nAs far as I see, and within the TfLite constraints, there are two possible approaches to this:</p>\n<p>1) Fix the sequence length during training (with truncating or padding as a preprocessing step).<br>\n-&gt; since sequences at inference time can be either longer or shorter that the fixed length at training time, this needs to be taken care of in the inference model as well<br>\n-&gt; Does anyone know how to do this??</p>\n<p>2) Train a truly variable sequence length model <br>\n-&gt; Does this work with the TfLite inference flow?<br>\n-&gt; If yes: has anyone figured out how to write a (parquet) Data loader to deal with variable length batches?</p>\n<p>Since these are technical issues that are very hard to overcome for TfLite newbies, it would be much appreciated if anyone (and especially the Google Tf or TfLite Gurus) could help here!</p>",
      "rawMarkdown": "Hi all,\n\nThe data we have (and the data the model should be able to use) are variable length sequences. \nAs far as I see, and within the TfLite constraints, there are two possible approaches to this:\n\n1) Fix the sequence length during training (with truncating or padding as a preprocessing step).\n-> since sequences at inference time can be either longer or shorter that the fixed length at training time, this needs to be taken care of in the inference model as well\n-> Does anyone know how to do this??\n\n2) Train a truly variable sequence length model \n-> Does this work with the TfLite inference flow?\n-> If yes: has anyone figured out how to write a (parquet) Data loader to deal with variable length batches?\n\nSince these are technical issues that are very hard to overcome for TfLite newbies, it would be much appreciated if anyone (and especially the Google Tf or TfLite Gurus) could help here!\n\n",
      "votes": 6
    },
    {
      "id": 2162939,
      "postDate": "2023-02-28T13:36:08.780Z",
      "content": "<p>Simplest method to \"interpolate\" frames is to us <a href=\"https://www.tensorflow.org/api_docs/python/tf/image/resize\" target=\"_blank\">tf.image.resize </a></p>\n<p><code>tf.image.resize(frames, [target_size, 543], method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)</code></p>\n<p>Funny enough it would work even if data is not an actual image!</p>\n<p>🖖</p>",
      "rawMarkdown": "Simplest method to \"interpolate\" frames is to us [tf.image.resize ](https://www.tensorflow.org/api_docs/python/tf/image/resize)\n\n`tf.image.resize(frames, [target_size, 543], method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)`\n\nFunny enough it would work even if data is not an actual image!\n\n🖖",
      "votes": 4
    },
    {
      "id": 2160645,
      "postDate": "2023-02-26T20:21:27.607Z",
      "content": "<p>One way is to set a number of frames (for example, 10) that you want and downsample or interpolate to upsample to get that number of frames as part of pre-processing the data. There are lots of ways to down and upsample. One way is just to linearly interpolate. </p>",
      "rawMarkdown": "One way is to set a number of frames (for example, 10) that you want and downsample or interpolate to upsample to get that number of frames as part of pre-processing the data. There are lots of ways to down and upsample. One way is just to linearly interpolate. ",
      "votes": 4
    },
    {
      "id": 2160381,
      "postDate": "2023-02-26T16:10:14.443Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 2162939,
      "author_name": "Mihai Cvasnievschi",
      "author_url": "",
      "post_date": "2023-02-28T13:36:08.780000",
      "content": "<p>Simplest method to \"interpolate\" frames is to us <a href=\"https://www.tensorflow.org/api_docs/python/tf/image/resize\" target=\"_blank\">tf.image.resize </a></p>\n<p><code>tf.image.resize(frames, [target_size, 543], method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)</code></p>\n<p>Funny enough it would work even if data is not an actual image!</p>\n<p>🖖</p>",
      "votes": 4,
      "replies": []
    },
    {
      "id": 2160645,
      "author_name": "Thad Starner",
      "author_url": "",
      "post_date": "2023-02-26T20:21:27.607000",
      "content": "<p>One way is to set a number of frames (for example, 10) that you want and downsample or interpolate to upsample to get that number of frames as part of pre-processing the data. There are lots of ways to down and upsample. One way is just to linearly interpolate. </p>",
      "votes": 4,
      "replies": []
    },
    {
      "id": 2160381,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-02-26T16:10:14.443000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2160306": "Hi all,\n\nThe data we have (and the data the model should be able to use) are variable length sequences. \nAs far as I see, and within the TfLite constraints, there are two possible approaches to this:\n\n1) Fix the sequence length during training (with truncating or padding as a preprocessing step).\n-> since sequences at inference time can be either longer or shorter that the fixed length at training time, this needs to be taken care of in the inference model as well\n-> Does anyone know how to do this??\n\n2) Train a truly variable sequence length model \n-> Does this work with the TfLite inference flow?\n-> If yes: has anyone figured out how to write a (parquet) Data loader to deal with variable length batches?\n\nSince these are technical issues that are very hard to overcome for TfLite newbies, it would be much appreciated if anyone (and especially the Google Tf or TfLite Gurus) could help here!\n\n",
    "2162939": "Simplest method to \"interpolate\" frames is to us [tf.image.resize ](https://www.tensorflow.org/api_docs/python/tf/image/resize)\n\n`tf.image.resize(frames, [target_size, 543], method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)`\n\nFunny enough it would work even if data is not an actual image!\n\n🖖",
    "2160645": "One way is to set a number of frames (for example, 10) that you want and downsample or interpolate to upsample to get that number of frames as part of pre-processing the data. There are lots of ways to down and upsample. One way is just to linearly interpolate. ",
    "2160381": ""
  }
}