Hello, Everyone
\nlet's say we want to do two steps in our preprocessing layer:
so my question is should we do these steps outside of our model?
\nif we make it a part of model, using a custom layer we cant train the model with batch_size>1 because arrays with different seq_len wont batch together, right?
\nand if we do it outside of the model, we cant inference unprocessed data
so what is the solution?
", "messages": [ { "id": "2212796", "postDate": "04/07/2023 05:18:45", "content": "Hello, Everyone
\nlet's say we want to do two steps in our preprocessing layer:
so my question is should we do these steps outside of our model?
\nif we make it a part of model, using a custom layer we cant train the model with batch_size>1 because arrays with different seq_len wont batch together, right?
\nand if we do it outside of the model, we cant inference unprocessed data
so what is the solution?
", "rawMarkdown": "Hello, Everyone \nlet's say we want to do two steps in our preprocessing layer:\n1. remove frames that both hands are missing\n2. fix the sequence length for all of our data to a certain number\n\nso my question is should we do these steps outside of our model?\nif we make it a part of model, using a custom layer we cant train the model with batch_size>1 because arrays with different seq_len wont batch together, right?\nand if we do it outside of the model, we cant inference unprocessed data\n\nso what is the solution?", "votes": null }, { "id": "2212869", "postDate": "04/07/2023 06:21:23", "content": "You could try to preprocess your data outside of the model and user-preprocessed data for training purposes. For the inference you will have to implement exactly the same preprocessing as a part of the model - only one sample in a batch is expected during submission.
\nAs an alternative approach, you could try exploiting tf.RagedTensor - they allow you to stack tensors of different shapes. It looks like it solves an issue you described above.
", "rawMarkdown": "You could try to preprocess your data outside of the model and user-preprocessed data for training purposes. For the inference you will have to implement exactly the same preprocessing as a part of the model - only one sample in a batch is expected during submission. \n\nAs an alternative approach, you could try exploiting tf.RagedTensor - they allow you to stack tensors of different shapes. It looks like it solves an issue you described above.", "votes": null }, { "id": "2213037", "postDate": "04/07/2023 09:15:17", "content": "Thanks, almost solved the issue
\nbut i have another problem now, see I'm implementing a custom preprocess layer (my first time ever) with (batch_size=None, seq_len, 543, 3) as input shape.
\nproblem is i use the input_shape to construct an NaN tensor, can't use input.shape apparently because it wont change when training different batch batch sizes it always stays locked to the shape of the first batch (i don't know the reason if you could explain I would be grateful), so need to use tf.shape(input) which cause this error:
\"NotImplementedError: Cannot convert a symbolic tf.Tensor (sequential_32/preprocess__layer_43/strided_slice:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported.\"
\ncan you help me this?
", "rawMarkdown": "Thanks, almost solved the issue\nbut i have another problem now, see I'm implementing a custom preprocess layer (my first time ever) with (batch_size=None, seq_len, 543, 3) as input shape.\nproblem is i use the input_shape to construct an NaN tensor, can't use input.shape apparently because it wont change when training different batch batch sizes it always stays locked to the shape of the first batch (i don't know the reason if you could explain I would be grateful), so need to use tf.shape(input) which cause this error:\n\n\"NotImplementedError: Cannot convert a symbolic tf.Tensor (sequential_32/preprocess__layer_43/strided_slice:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported.\"\n\ncan you help me this?", "votes": null }, { "id": "2213068", "postDate": "04/07/2023 09:30:51", "content": "I am not sure whether I did you correctly, but I would describe an approach that works for me.
\nUsually, I divide a preprocessing layer and model itself during training. So, my model works with arbitrary batch size. After the model was trained, I create a new model. This model now consists of preprocessing layer and a trained previous model. Here is code snippet I am using:
\n (tf.keras.Model):\n \n\n ():\n \n (TFLiteModel, self).__init__()\n\n \n self.prep_inputs = FeatureGen()\n self.asl_model = asl_model\n\n\n ():\n \n x = self.prep_inputs(tf.cast(inputs, dtype=tf.float32))\n x = tf.expand_dims(x, axis=)\n outputs = self.asl_model(x)[, :]\nNote, that here we expect to consume samples without batches and expand dims manually. We also slice prediction along batch size (take the first and actually single one). Hope, that would help you
", "rawMarkdown": "I am not sure whether I did you correctly, but I would describe an approach that works for me.\n\nUsually, I divide a preprocessing layer and model itself during training. So, my model works with arbitrary batch size. After the model was trained, I create a new model. This model now consists of preprocessing layer and a trained previous model. Here is code snippet I am using:\n\n\n```python\nclass TFLiteModel(tf.keras.Model):\n \"\"\"\n TensorFlow Lite model that takes input tensors and applies:\n – a preprocessing model\n – the ASL model \n \"\"\"\n\n def __init__(self, asl_model):\n \"\"\"\n Initializes the TFLiteModel with the specified feature generation model and main model.\n \"\"\"\n super(TFLiteModel, self).__init__()\n\n # Load the feature generation and main models\n self.prep_inputs = FeatureGen()\n self.asl_model = asl_model\n \n @tf.function(input_signature=[tf.TensorSpec(shape=[None, 543, 3], dtype=tf.float32, name='inputs')])\n def call(self, inputs):\n \"\"\"\n Applies the feature generation model and main model to the input tensors.\n\n Args:\n inputs: Input tensor with shape [batch_size, 543, 3].\n\n Returns:\n A dictionary with a single key 'outputs' and corresponding output tensor.\n \"\"\"\n x = self.prep_inputs(tf.cast(inputs, dtype=tf.float32))\n x = tf.expand_dims(x, axis=0)\n outputs = self.asl_model(x)[0, :]\n```\n\n\nNote, that here we expect to consume samples without batches and expand dims manually. We also slice prediction along batch size (take the first and actually single one). Hope, that would help you", "votes": null }, { "id": "2213114", "postDate": "04/07/2023 10:07:35", "content": "Thank you very much, I get what you saying, very useful tip
\nbut lets say we want our preprocess layer has a dynamic batch size so that it can preprocess in different batches what should we do then
\nhere is my notebook if you can tell me what is wrong with it I'd be very grateful
\n\nhttps://www.kaggle.com/code/amirhosseintondari/slr-v1
Try to specify None as a batch size
", "rawMarkdown": "Try to specify None as a batch size", "votes": null }, { "id": "2213246", "postDate": "04/07/2023 12:46:39", "content": "Then i get the following type error:
\n'in user code:
\nFile \"/tmp/ipykernel_27/1106163312.py\", line 33, in call *\n nantns = tf.constant(float('nan'), shape=(in_shape[0], 1, 62, 3))\nFile \"<__array_function__ internals>\", line 6, in prod\n\nFile \"/opt/conda/lib/python3.7/site-packages/numpy/core/fromnumeric.py\", line 3052, in prod\n keepdims=keepdims, initial=initial, where=where)\nFile \"/opt/conda/lib/python3.7/site-packages/numpy/core/fromnumeric.py\", line 86, in _wrapreduction\n return ufunc.reduce(obj, axis, dtype, out, **passkwargs)\n\nTypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'\nCall arguments received by layer 'preprocess__layer_15' (type Preprocess_Layer):
\n • x=tf.Tensor(shape=(None, 105, 543, 3), dtype=float32).'
lets just forget about batch_size for a moment, lets say we want our model to take a tensor of any size and output a nan tensor with the same dimensions, but the catch is that each time we call the model to predict, the input has different shape.
\nthe problem lies when we call the model to build or fit, the input dims get fixed i want it to be dynamic
\n\n• x=tf.Tensor(shape=(None, 105, 543, 3), dtype=float32).'
\n
If I am not mistaken, tensorflow works with '-1' instead of None
", "rawMarkdown": "> • x=tf.Tensor(shape=(None, 105, 543, 3), dtype=float32).'\n\nIf I am not mistaken, tensorflow works with '-1' instead of None", "votes": null }, { "id": "2213387", "postDate": "04/07/2023 14:43:00", "content": "Actually, graph mode allows dealing with arbitrary shapes. For instance, take a look at this function decorated with @tf.function. It works in graph mode perfectly
\n\n ():\n x**\n\na = tf.constant([, , ])\n(foo(a))\n\nb = tf.constant([[, ], [, ]])\n(foo(b))```\n\nIt prints the following:\ntf.Tensor([1 4 9], shape=(3,), dtype=int32)
\ntf.Tensor(
\n[[1 9]
\n [0 1]], shape=(2, 2), dtype=int32)```
Unfortunately, I lost permission to see your notebook, but I can confirm that the problem is not in dynamic shape. Every time tensorflow sees a new shape, it creates a new graph for that shape by default (may be configured with @tf.function arguments)
", "rawMarkdown": "Actually, graph mode allows dealing with arbitrary shapes. For instance, take a look at this function decorated with @tf.function. It works in graph mode perfectly\n\n```python\n@tf.function\ndef foo(x):\n return x**2\n\na = tf.constant([1, 2, 3])\nprint(foo(a))\n\nb = tf.constant([[1, 3], [0, 1]])\nprint(foo(b))```\n\nIt prints the following:\n\n```tf.Tensor([1 4 9], shape=(3,), dtype=int32)\ntf.Tensor(\n[[1 9]\n [0 1]], shape=(2, 2), dtype=int32)```\n\nUnfortunately, I lost permission to see your notebook, but I can confirm that the problem is not in dynamic shape. Every time tensorflow sees a new shape, it creates a new graph for that shape by default (may be configured with @tf.function arguments)", "votes": null }, { "id": "2213406", "postDate": "04/07/2023 14:59:23", "content": "wow, thanks a lot fixed my issue, it was bothering me for a while
", "rawMarkdown": "wow, thanks a lot fixed my issue, it was bothering me for a while", "votes": null } ], "comments": [ { "id": 2212869, "author_name": "meowmeowmeowmeowmeow", "author_url": "", "post_date": "04/07/2023 06:21:23", "content": "You could try to preprocess your data outside of the model and user-preprocessed data for training purposes. For the inference you will have to implement exactly the same preprocessing as a part of the model - only one sample in a batch is expected during submission.
\nAs an alternative approach, you could try exploiting tf.RagedTensor - they allow you to stack tensors of different shapes. It looks like it solves an issue you described above.
", "votes": null, "replies": [ { "id": 2213037, "author_name": "amirhosseintondari", "author_url": "", "post_date": "04/07/2023 09:15:17", "content": "Thanks, almost solved the issue
\nbut i have another problem now, see I'm implementing a custom preprocess layer (my first time ever) with (batch_size=None, seq_len, 543, 3) as input shape.
\nproblem is i use the input_shape to construct an NaN tensor, can't use input.shape apparently because it wont change when training different batch batch sizes it always stays locked to the shape of the first batch (i don't know the reason if you could explain I would be grateful), so need to use tf.shape(input) which cause this error:
\"NotImplementedError: Cannot convert a symbolic tf.Tensor (sequential_32/preprocess__layer_43/strided_slice:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported.\"
\ncan you help me this?
", "votes": null, "replies": [ { "id": 2213068, "author_name": "meowmeowmeowmeowmeow", "author_url": "", "post_date": "04/07/2023 09:30:51", "content": "I am not sure whether I did you correctly, but I would describe an approach that works for me.
\nUsually, I divide a preprocessing layer and model itself during training. So, my model works with arbitrary batch size. After the model was trained, I create a new model. This model now consists of preprocessing layer and a trained previous model. Here is code snippet I am using:
\n (tf.keras.Model):\n \n\n ():\n \n (TFLiteModel, self).__init__()\n\n \n self.prep_inputs = FeatureGen()\n self.asl_model = asl_model\n\n\n ():\n \n x = self.prep_inputs(tf.cast(inputs, dtype=tf.float32))\n x = tf.expand_dims(x, axis=)\n outputs = self.asl_model(x)[, :]\nNote, that here we expect to consume samples without batches and expand dims manually. We also slice prediction along batch size (take the first and actually single one). Hope, that would help you
", "votes": null, "replies": [ { "id": 2213114, "author_name": "amirhosseintondari", "author_url": "", "post_date": "04/07/2023 10:07:35", "content": "Thank you very much, I get what you saying, very useful tip
\nbut lets say we want our preprocess layer has a dynamic batch size so that it can preprocess in different batches what should we do then
\nhere is my notebook if you can tell me what is wrong with it I'd be very grateful
\n\nhttps://www.kaggle.com/code/amirhosseintondari/slr-v1
Try to specify None as a batch size
", "votes": null, "replies": [ { "id": 2213246, "author_name": "amirhosseintondari", "author_url": "", "post_date": "04/07/2023 12:46:39", "content": "Then i get the following type error:
\n'in user code:
\nFile \"/tmp/ipykernel_27/1106163312.py\", line 33, in call *\n nantns = tf.constant(float('nan'), shape=(in_shape[0], 1, 62, 3))\nFile \"<__array_function__ internals>\", line 6, in prod\n\nFile \"/opt/conda/lib/python3.7/site-packages/numpy/core/fromnumeric.py\", line 3052, in prod\n keepdims=keepdims, initial=initial, where=where)\nFile \"/opt/conda/lib/python3.7/site-packages/numpy/core/fromnumeric.py\", line 86, in _wrapreduction\n return ufunc.reduce(obj, axis, dtype, out, **passkwargs)\n\nTypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'\nCall arguments received by layer 'preprocess__layer_15' (type Preprocess_Layer):
\n • x=tf.Tensor(shape=(None, 105, 543, 3), dtype=float32).'
lets just forget about batch_size for a moment, lets say we want our model to take a tensor of any size and output a nan tensor with the same dimensions, but the catch is that each time we call the model to predict, the input has different shape.
\nthe problem lies when we call the model to build or fit, the input dims get fixed i want it to be dynamic
Actually, graph mode allows dealing with arbitrary shapes. For instance, take a look at this function decorated with @tf.function. It works in graph mode perfectly
\n\n ():\n x**\n\na = tf.constant([, , ])\n(foo(a))\n\nb = tf.constant([[, ], [, ]])\n(foo(b))```\n\nIt prints the following:\ntf.Tensor([1 4 9], shape=(3,), dtype=int32)
\ntf.Tensor(
\n[[1 9]
\n [0 1]], shape=(2, 2), dtype=int32)```
Unfortunately, I lost permission to see your notebook, but I can confirm that the problem is not in dynamic shape. Every time tensorflow sees a new shape, it creates a new graph for that shape by default (may be configured with @tf.function arguments)
", "votes": null, "replies": [ { "id": 2213406, "author_name": "amirhosseintondari", "author_url": "", "post_date": "04/07/2023 14:59:23", "content": "wow, thanks a lot fixed my issue, it was bothering me for a while
", "votes": null, "replies": [] } ] } ] }, { "id": 2213383, "author_name": "meowmeowmeowmeowmeow", "author_url": "", "post_date": "04/07/2023 14:38:50", "content": "\n\n• x=tf.Tensor(shape=(None, 105, 543, 3), dtype=float32).'
\n
If I am not mistaken, tensorflow works with '-1' instead of None
", "votes": null, "replies": [] } ] } ] } ] } ] } ] } ] } ], "raw_markdown_by_id": { "2212796": "Hello, Everyone \nlet's say we want to do two steps in our preprocessing layer:\n1. remove frames that both hands are missing\n2. fix the sequence length for all of our data to a certain number\n\nso my question is should we do these steps outside of our model?\nif we make it a part of model, using a custom layer we cant train the model with batch_size>1 because arrays with different seq_len wont batch together, right?\nand if we do it outside of the model, we cant inference unprocessed data\n\nso what is the solution?", "2212869": "You could try to preprocess your data outside of the model and user-preprocessed data for training purposes. For the inference you will have to implement exactly the same preprocessing as a part of the model - only one sample in a batch is expected during submission. \n\nAs an alternative approach, you could try exploiting tf.RagedTensor - they allow you to stack tensors of different shapes. It looks like it solves an issue you described above.", "2213037": "Thanks, almost solved the issue\nbut i have another problem now, see I'm implementing a custom preprocess layer (my first time ever) with (batch_size=None, seq_len, 543, 3) as input shape.\nproblem is i use the input_shape to construct an NaN tensor, can't use input.shape apparently because it wont change when training different batch batch sizes it always stays locked to the shape of the first batch (i don't know the reason if you could explain I would be grateful), so need to use tf.shape(input) which cause this error:\n\n\"NotImplementedError: Cannot convert a symbolic tf.Tensor (sequential_32/preprocess__layer_43/strided_slice:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported.\"\n\ncan you help me this?", "2213068": "I am not sure whether I did you correctly, but I would describe an approach that works for me.\n\nUsually, I divide a preprocessing layer and model itself during training. So, my model works with arbitrary batch size. After the model was trained, I create a new model. This model now consists of preprocessing layer and a trained previous model. Here is code snippet I am using:\n\n\n```python\nclass TFLiteModel(tf.keras.Model):\n \"\"\"\n TensorFlow Lite model that takes input tensors and applies:\n – a preprocessing model\n – the ASL model \n \"\"\"\n\n def __init__(self, asl_model):\n \"\"\"\n Initializes the TFLiteModel with the specified feature generation model and main model.\n \"\"\"\n super(TFLiteModel, self).__init__()\n\n # Load the feature generation and main models\n self.prep_inputs = FeatureGen()\n self.asl_model = asl_model\n \n @tf.function(input_signature=[tf.TensorSpec(shape=[None, 543, 3], dtype=tf.float32, name='inputs')])\n def call(self, inputs):\n \"\"\"\n Applies the feature generation model and main model to the input tensors.\n\n Args:\n inputs: Input tensor with shape [batch_size, 543, 3].\n\n Returns:\n A dictionary with a single key 'outputs' and corresponding output tensor.\n \"\"\"\n x = self.prep_inputs(tf.cast(inputs, dtype=tf.float32))\n x = tf.expand_dims(x, axis=0)\n outputs = self.asl_model(x)[0, :]\n```\n\n\nNote, that here we expect to consume samples without batches and expand dims manually. We also slice prediction along batch size (take the first and actually single one). Hope, that would help you", "2213114": "Thank you very much, I get what you saying, very useful tip\nbut lets say we want our preprocess layer has a dynamic batch size so that it can preprocess in different batches what should we do then\nhere is my notebook if you can tell me what is wrong with it I'd be very grateful\n[\nhttps://www.kaggle.com/code/amirhosseintondari/slr-v1](url)", "2213182": "Try to specify None as a batch size", "2213246": "Then i get the following type error:\n\n'in user code:\n\n File \"/tmp/ipykernel_27/1106163312.py\", line 33, in call *\n nantns = tf.constant(float('nan'), shape=(in_shape[0], 1, 62, 3))\n File \"<__array_function__ internals>\", line 6, in prod\n \n File \"/opt/conda/lib/python3.7/site-packages/numpy/core/fromnumeric.py\", line 3052, in prod\n keepdims=keepdims, initial=initial, where=where)\n File \"/opt/conda/lib/python3.7/site-packages/numpy/core/fromnumeric.py\", line 86, in _wrapreduction\n return ufunc.reduce(obj, axis, dtype, out, **passkwargs)\n\n TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'\n\n\nCall arguments received by layer 'preprocess__layer_15' (type Preprocess_Layer):\n • x=tf.Tensor(shape=(None, 105, 543, 3), dtype=float32).'", "2213258": "lets just forget about batch_size for a moment, lets say we want our model to take a tensor of any size and output a nan tensor with the same dimensions, but the catch is that each time we call the model to predict, the input has different shape.\nthe problem lies when we call the model to build or fit, the input dims get fixed i want it to be dynamic", "2213383": "> • x=tf.Tensor(shape=(None, 105, 543, 3), dtype=float32).'\n\nIf I am not mistaken, tensorflow works with '-1' instead of None", "2213387": "Actually, graph mode allows dealing with arbitrary shapes. For instance, take a look at this function decorated with @tf.function. It works in graph mode perfectly\n\n```python\n@tf.function\ndef foo(x):\n return x**2\n\na = tf.constant([1, 2, 3])\nprint(foo(a))\n\nb = tf.constant([[1, 3], [0, 1]])\nprint(foo(b))```\n\nIt prints the following:\n\n```tf.Tensor([1 4 9], shape=(3,), dtype=int32)\ntf.Tensor(\n[[1 9]\n [0 1]], shape=(2, 2), dtype=int32)```\n\nUnfortunately, I lost permission to see your notebook, but I can confirm that the problem is not in dynamic shape. Every time tensorflow sees a new shape, it creates a new graph for that shape by default (may be configured with @tf.function arguments)", "2213406": "wow, thanks a lot fixed my issue, it was bothering me for a while" }, "source": "meta" }