Hey all! I'm Paul, a developer advocate with the TensorFlow team. I wanted to make a discussion thread here for all TensorFlow Lite related questions so we can follow along in a centralized place and get the correct people around for sharing their knowledge :)
", "messages": [ { "id": 2157047, "postDate": "2023-02-23T17:38:21.647Z", "content": "Hey all! I'm Paul, a developer advocate with the TensorFlow team. I wanted to make a discussion thread here for all TensorFlow Lite related questions so we can follow along in a centralized place and get the correct people around for sharing their knowledge :)
", "rawMarkdown": "Hey all! I'm Paul, a developer advocate with the TensorFlow team. I wanted to make a discussion thread here for all TensorFlow Lite related questions so we can follow along in a centralized place and get the correct people around for sharing their knowledge :)", "votes": 26 }, { "id": 2160198, "postDate": "2023-02-26T13:13:00.060Z", "content": "\n\nYour model must also require less than 40 MB in memory
\n
How do we calculate the in-memory size of a model? What specific checks does the submission evaluation run - so that we can pre-check our models and avoid wasted submissions.
", "rawMarkdown": "> Your model must also require less than 40 MB in memory\n\nHow do we calculate the in-memory size of a model? What specific checks does the submission evaluation run - so that we can pre-check our models and avoid wasted submissions.", "votes": 11, "replies": [ { "id": 2169859, "postDate": "2023-03-05T13:58:28.253Z", "content": "This is my rule of thumb:
\nCalculate the memory usage of the model by multiplying the number of parameters by the number of bytes required to store each parameter. For example, if your model has 10 million parameters and each parameter requires 4 bytes, the total memory usage would be 40 MB.
\nNote that the actual memory usage of the model may differ from this calculated value due to various factors such as the use of GPU memory, data type of the parameters, and any compression or optimization techniques used.
\nI’d also note that it appears more permissive than the above because I’ve definitely submitted models that are 10M+ that have succeeded.
\nHope that helps!
", "rawMarkdown": "This is my rule of thumb:\n\nCalculate the memory usage of the model by multiplying the number of parameters by the number of bytes required to store each parameter. For example, if your model has 10 million parameters and each parameter requires 4 bytes, the total memory usage would be 40 MB.\n\nNote that the actual memory usage of the model may differ from this calculated value due to various factors such as the use of GPU memory, data type of the parameters, and any compression or optimization techniques used.\n\nI’d also note that it appears more permissive than the above because I’ve definitely submitted models that are 10M+ that have succeeded.\n\nHope that helps!", "votes": 6 } ] }, { "id": 2162315, "postDate": "2023-02-28T05:59:02.477Z", "content": "I recently started learning Tensorflow and wish to confirm that my understanding is correct:
\nIs there a list somewhere that tells us what layers are compatible with TFlite? Since we are dealing with sequences of data, I tried using LSTM and GRU, but both don't seem to be compatible with TFlite. It technically runs, but when I convert to TFlite I get this warning:
\nWARNING:absl:Found untraced functions such as _update_step_xla, lstm_cell_1_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_2_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 5). These functions will not be directly callable after loading.
\nThe accuracy when using the TFlite model is much lower than when using regular TF model. Assuming it's because GRU and LSTM are not compatible with TFlite?
", "rawMarkdown": "Is there a list somewhere that tells us what layers are compatible with TFlite? Since we are dealing with sequences of data, I tried using LSTM and GRU, but both don't seem to be compatible with TFlite. It technically runs, but when I convert to TFlite I get this warning: \n\nWARNING:absl:Found untraced functions such as _update_step_xla, lstm_cell_1_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_2_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 5). These functions will not be directly callable after loading.\n\nThe accuracy when using the TFlite model is much lower than when using regular TF model. Assuming it's because GRU and LSTM are not compatible with TFlite?", "votes": 3 }, { "id": 2157188, "postDate": "2023-02-23T19:05:00.070Z", "content": "As, I am at starting state of learning, I don't known weather it will complete or not. But for me this project seams to be a great one and I will try to learn at least something while working on this.
\nthank you for this amazing project
", "rawMarkdown": "As, I am at starting state of learning, I don't known weather it will complete or not. But for me this project seams to be a great one and I will try to learn at least something while working on this.\n\n\nthank you for this amazing project\n", "votes": 3 }, { "id": 2158548, "postDate": "2023-02-25T00:32:07.537Z", "content": "\n\nYour model must also require less than 40 MB in memory and perform inference with less than 100 milliseconds of latency per video.
\n
How should we calculate the inference latency for each video on the validation set? Should we calculate the time between reading data from disk and getting inference results? Because I assume we can't get all the data to be read into RAM or graphics memory
", "rawMarkdown": ">Your model must also require less than 40 MB in memory and perform inference with less than 100 milliseconds of latency per video. \n\nHow should we calculate the inference latency for each video on the validation set? Should we calculate the time between reading data from disk and getting inference results? Because I assume we can't get all the data to be read into RAM or graphics memory", "votes": 4, "replies": [ { "id": 2160593, "postDate": "2023-02-26T19:46:55.430Z", "content": "It talks about the storage and runtime your trained model takes for a given input. Hope this clarifies your query!
", "rawMarkdown": "It talks about the storage and runtime your trained model takes for a given input. Hope this clarifies your query!", "votes": -3 }, { "id": 2169858, "postDate": "2023-03-05T13:56:41.720Z", "content": "For inference timing you can time the inference code that they provide in a loop and time it. If your using a variable time series, I would do the 75% upper quantile of number of frames to mimic a slightly longer than average runtime. That should give you a bit of buffer.
\nFor size, calculate the memory usage of the model by multiplying the number of parameters by the number of bytes required to store each parameter. For example, if your model has 10 million parameters and each parameter requires 4 bytes, the total memory usage would be 40 MB.
\nNote that the actual memory usage of the model may differ from this calculated value due to various factors such as the use of GPU memory, data type of the parameters, and any compression or optimization techniques used.
Hope that helps!
", "rawMarkdown": "For inference timing you can time the inference code that they provide in a loop and time it. If your using a variable time series, I would do the 75% upper quantile of number of frames to mimic a slightly longer than average runtime. That should give you a bit of buffer.\n\nFor size, calculate the memory usage of the model by multiplying the number of parameters by the number of bytes required to store each parameter. For example, if your model has 10 million parameters and each parameter requires 4 bytes, the total memory usage would be 40 MB.\nNote that the actual memory usage of the model may differ from this calculated value due to various factors such as the use of GPU memory, data type of the parameters, and any compression or optimization techniques used.\n\nHope that helps!" } ] }, { "id": 2157243, "postDate": "2023-02-23T20:35:48.100Z", "content": "what is the best tools/way to convert Pytorch model to tflite?
", "rawMarkdown": "what is the best tools/way to convert Pytorch model to tflite?", "votes": 1, "replies": [ { "id": 2157381, "postDate": "2023-02-24T01:03:09.540Z", "content": "We see many PyTorch developers who deploy via TensorFlow Lite use a path of PyTorch -> ONNX -> TensorFlow -> TensorFlow Lite. Two community posts demonstrating this path are https://medium.com/geekculture/converting-yolo-v7-to-tensorflow-lite-for-mobile-deployment-ebc1103e8d1e and https://github.com/sayakpaul/Adventures-in-TensorFlow-Lite/blob/master/TUNIT_Conversion_to_TF_Lite.ipynb
", "rawMarkdown": "We see many PyTorch developers who deploy via TensorFlow Lite use a path of PyTorch -> ONNX -> TensorFlow -> TensorFlow Lite. Two community posts demonstrating this path are https://medium.com/geekculture/converting-yolo-v7-to-tensorflow-lite-for-mobile-deployment-ebc1103e8d1e and https://github.com/sayakpaul/Adventures-in-TensorFlow-Lite/blob/master/TUNIT_Conversion_to_TF_Lite.ipynb", "votes": 9 } ] }, { "id": 2157385, "postDate": "2023-02-24T01:04:14.190Z", "content": "\n\nThis competition requires submissions to be made in the form of TensorFlow Lite models. You are welcome to train your model using the framework of your choice as long as you convert the model checkpoint into the tflite format prior to submission. Please see the evaluation page for details.
\n
What if we did not use TensorFlow Lite models? We will be removed from Leaderboard?
", "rawMarkdown": "> This competition requires submissions to be made in the form of TensorFlow Lite models. You are welcome to train your model using the framework of your choice as long as you convert the model checkpoint into the tflite format prior to submission. Please see the evaluation page for details.\n\nWhat if we did not use TensorFlow Lite models? We will be removed from Leaderboard?", "replies": [ { "id": 2157387, "postDate": "2023-02-24T01:07:14.163Z", "content": "Your submission will fail without a TF Lite model.
", "rawMarkdown": "Your submission will fail without a TF Lite model.", "votes": 3, "replies": [ { "id": 2216731, "postDate": "2023-04-10T09:52:33.367Z", "content": "Cool, thank you!
", "rawMarkdown": "Cool, thank you!" } ] } ] }, { "id": 2217824, "postDate": "2023-04-11T07:39:54.293Z", "content": "Thanks for you amazing code, it helps me a lot
", "rawMarkdown": "Thanks for you amazing code, it helps me a lot", "votes": -2 }, { "id": 2232348, "postDate": "2023-04-24T08:31:08.840Z", "content": "very good.this code help me a lot
", "rawMarkdown": "very good.this code help me a lot" }, { "id": 2226249, "postDate": "2023-04-18T19:17:47.850Z", "content": "Hello
\nI receive error message:
error: op neither a custom op nor a flex op\nerror: failed converting: : \nSome ops are supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select \nTF Select ops: Conv2D\nDetails:\n tf.Conv2D(tensor<*xf32>, tensor<2x2x3x32xf32>) -> (tensor<?x?x?x32xf32>) : {data_format = , device = , dilations = [, , , ], explicit_paddings = [], padding = , strides = [, , , ], use_cudnn_on_gpu = true}\nWhy it's happens? According https://www.tensorflow.org/mlir/tfl_ops?hl=ru#tflconv_2d_mlirtflconv2dop conv_2d layers are supported
", "rawMarkdown": "Hello\nI receive error message:\n\n```python\nerror: 'tf.Conv2D' op is neither a custom op nor a flex op\nerror: failed while converting: 'main': \nSome ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select \nTF Select ops: Conv2D\nDetails:\n tf.Conv2D(tensor<*xf32>, tensor<2x2x3x32xf32>) -> (tensor) : {data_format = \"NHWC\", device = \"\", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = \"SAME\", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}\n```\nWhy it's happens? According https://www.tensorflow.org/mlir/tfl_ops?hl=ru#tflconv_2d_mlirtflconv2dop conv_2d layers are supported" }, { "id": 2218660, "postDate": "2023-04-11T22:27:55.273Z", "content": "TFlite models seem to grow in size very quickly for just a few more features in the data. Are there standard ways to shrink the size?
", "rawMarkdown": "TFlite models seem to grow in size very quickly for just a few more features in the data. Are there standard ways to shrink the size? " }, { "id": 2198405, "postDate": "2023-03-27T01:22:30.350Z", "content": "Several people have already mentioned the topic of \"TensorFlow operators\".
\nAs far as I can confirm, the current submission environment does not support TensorFlow operators, I would expect.
\nI checked on the Kaggle notebook and it is possible to support TensorFlow operators in tflite_runtime by importing tensorflow as follows
@@ -1,3 +1,4 @@\n+import tensorflow\n import tflite_runtime.interpreter as tflite\n interpreter = tflite.Interpreter(model_path)\nIt seems that the introduction of TensorFlow operators is not impractical for device implementations.
\nhttps://www.tensorflow.org/lite/guide/ops_select
Is there any possibility of this modification to the submission environment?
\nI would be very happy if TensorFlow operators were introduced, as I think it would make it easier to implement more complex models and increase our chances of achieving greater results.
\nI welcome any comments on implementation issues on the device, concerns about competitiveness, etc.
", "rawMarkdown": "Several people have already mentioned the topic of \"TensorFlow operators\".\nAs far as I can confirm, the current submission environment does not support TensorFlow operators, I would expect.\nI checked on the Kaggle notebook and it is possible to support TensorFlow operators in tflite_runtime by importing tensorflow as follows\n\n```\n@@ -1,3 +1,4 @@\n+import tensorflow\n import tflite_runtime.interpreter as tflite\n interpreter = tflite.Interpreter(model_path)\n```\n\nIt seems that the introduction of TensorFlow operators is not impractical for device implementations.\nhttps://www.tensorflow.org/lite/guide/ops_select\n\nIs there any possibility of this modification to the submission environment?\n\nI would be very happy if TensorFlow operators were introduced, as I think it would make it easier to implement more complex models and increase our chances of achieving greater results.\n\nI welcome any comments on implementation issues on the device, concerns about competitiveness, etc.\n", "replies": [ { "id": 2199328, "postDate": "2023-03-27T16:27:59.200Z", "content": "We're deliberately excluding TensorFlow operators as the goal of the competition is to generate models that could potentially run on cell phones.
", "rawMarkdown": "We're deliberately excluding TensorFlow operators as the goal of the competition is to generate models that could potentially run on cell phones.", "votes": 1 } ] }, { "id": 2189855, "postDate": "2023-03-20T20:52:02.770Z", "content": "Hello, guys. I faced an issue with tf.data pipeline working in graph mode. I need to edit the tensor somehow, but I cannot do so. I understand that tf.Tensor is immutable and supposed to work in a functional style. So, I am creating tf.Variable from it and that works well in eager execution mode. But tf.data pipeline uses graph mode and makes tf.Variable creation in such a way impossible.
\nHere is a minimalistic code, which reproduce my issue:
\n\nimport tensorflow as tf
\nx = tf.constant([1,2,3.0])
\n
\n ds = tf.data.Dataset.from_tensors(x)def test(x):
\n
\n with tf.init_scope():
\n v = tf.Variable(x)ds_x = ds.map(test)
\n
\n next(iter(ds_x))
Here is an error I get
\n`
\nValueError: in user code:
File \"/tmp/ipykernel_3517/140449158.py\", line 6, in test *\n v = tf.Variable(x)\n\nValueError: Argument `initial_value` (Tensor(\"args_0:0\", shape=(3,), dtype=float32)) could not be lifted out of a `tf.function`. (Tried to create variable with name='None'). To avoid this error, when constructing `tf.Variable`s inside of `tf.function` you can create the `initial_value` tensor in a `tf.init_scope` or pass a callable `initial_value` (e.g., `tf.Variable(lambda : tf.truncated_normal([10, 40]))`). Please file a feature request if this restriction inconveniences you.\n`
\nBest to my understanding, I cannot create a tf.Variable which depends on input tensor x because of static execution graph. But how do I change a part of input tensor x, where the decision of that part is depends on x content?
Looking forward to your advice. Thank you in advance
", "rawMarkdown": "Hello, guys. I faced an issue with tf.data pipeline working in graph mode. I need to edit the tensor somehow, but I cannot do so. I understand that tf.Tensor is immutable and supposed to work in a functional style. So, I am creating tf.Variable from it and that works well in eager execution mode. But tf.data pipeline uses graph mode and makes tf.Variable creation in such a way impossible. \nHere is a minimalistic code, which reproduce my issue:\n\n>import tensorflow as tf\n>\n>x = tf.constant([1,2,3.0])\n>ds = tf.data.Dataset.from_tensors(x)\n>\n>def test(x):\n> with tf.init_scope():\n> v = tf.Variable(x)\n>\n>ds_x = ds.map(test)\n>next(iter(ds_x))\n\nHere is an error I get\n\n`\nValueError: in user code:\n\n File \"/tmp/ipykernel_3517/140449158.py\", line 6, in test *\n v = tf.Variable(x)\n\n ValueError: Argument `initial_value` (Tensor(\"args_0:0\", shape=(3,), dtype=float32)) could not be lifted out of a `tf.function`. (Tried to create variable with name='None'). To avoid this error, when constructing `tf.Variable`s inside of `tf.function` you can create the `initial_value` tensor in a `tf.init_scope` or pass a callable `initial_value` (e.g., `tf.Variable(lambda : tf.truncated_normal([10, 40]))`). Please file a feature request if this restriction inconveniences you.\n`\n\nBest to my understanding, I cannot create a tf.Variable which depends on input tensor `x` because of static execution graph. But how do I change a part of input tensor `x`, where the decision of that part is depends on `x` content?\n\nLooking forward to your advice. Thank you in advance", "replies": [ { "id": 2190236, "postDate": "2023-03-21T06:04:22.103Z", "content": "Hi Mykola,
\nThis is the code snippet I was sent for your issue:
\nimport tensorflow as tf\n\nx = tf.constant([1,2,3,4,5,6])\nds = tf.data.Dataset.from_tensor_slices(x)\n\n@tf.function\ndef test(x):\n if x > 2:\n return x**2\n else:\n return x\n\nds_x = ds.map(test)\n\nfor x in iter(ds_x):\n print(x)\nYou can also find a lot of examples for tf.data here: https://www.tensorflow.org/guide/data
", "rawMarkdown": "Hi Mykola,\n\nThis is the code snippet I was sent for your issue:\n\n```\nimport tensorflow as tf\n\nx = tf.constant([1,2,3,4,5,6])\nds = tf.data.Dataset.from_tensor_slices(x)\n\n@tf.function\ndef test(x):\n if x > 2:\n return x**2\n else:\n return x\n\nds_x = ds.map(test)\n\nfor x in iter(ds_x):\n print(x)\n```\n\nYou can also find a lot of examples for tf.data here: https://www.tensorflow.org/guide/data", "votes": 1, "replies": [ { "id": 2190448, "postDate": "2023-03-21T08:40:22.010Z", "content": "I am not sure, whether this solves my problem, but further investigation gives me the following intuition and almost the solution (I am still in progress now). I would like to ask you whether my thoughts are correct or if am I wrong in so something.
\nThe goal is to calculate the mean of the input vector by segments. The actual segments depend on the content of the input vector.
\n ():\n ...\n segments \nAnd now I am going to iterate over each segments and calculate mean
\n ():\n segments = get_segments(x)\n mean_list = []\n i segments: \n mean_list.append(tf.math.reduce_mean(x[i[]:i[])) \n\n means = tf.stack(mean_list) \nThis works pretty well in eager mode, but fails if I decorate it with @tf.function. That makes sense. Creating a mutable list is a violation of functional programming, thus we could not build a static computational graph. In fact, a list works well, if I create it with a predefined length (like, mean_list = [tf.constant(2), tf.math.reduce_mean(x[a:b]), ...]). I guess TensorFlow graph compiler is smart enough to understand that list in that case is not going to be mutable ever.
Regarding the task of calculating mean over segments currently, I found a possible solution with tf.map_fn. I discover a way to pass multiple arguments to fn using tf.map_fn (via tuples) - vector x and segment i in my case. This way enforces me to have the same shape over the first axis. So, I need to tile an x vector N times, in order to have the same length first axis - [N, None] for x and [N, 2] for segments. Hope, that would resolve my current issue.
Taking the opportunity, I would like to ask you the following questions:
\nAm I right, that all code under tf.function should be purely functional?
Does my solution with tf.map_fn the best one for the task described above?
Doesn't the tf.data pipeline support eager execution mode?
Sorry, I some of the questions are trivial, but I spent a loot of time starting from the first release of TF2.0 on this. Thank you in advance!
The solution with tf.map_fn works, until fn returns tensors of different shapes. I want to apply the following function
():\n x = arg[]\n _ = arg[]\n a = r[]\n b = r[]\n x[a:b]*x[] \n\n ():\n segments = get_segments(x)\n a = tf.tile(x, [(segments)]) \n a = tf.reshape(a, [(segments), -])\n c = tf.map_fn(take_it, (a,segments), fn_output_signature=tf.int64)\n\n means = tf.stack(mean_list) \nLet say x is a tensor [0,1,2,3,4,5,6,7,8,9] and get_segments(x) returns the following
[[0, 3],\n [3, 7],\n [7, 10]]\nI expect to get [0,0, 4,6,8,10, 49,63] as a result, but it is impossible since take_it returns tensors of different shapes. Now I get the following error:
2023-03-21 09:52:22.638552: W tensorflow/core/framework/op_kernel.cc:1830] OP_REQUIRES failed at list_kernels.h:206 : INVALID_ARGUMENT: PartialTensorShape: Incompatible shapes during merge: [2] vs. [4]\nCould anyone give me an idea of how to solve my problem with @tf.function approach?
It was a little bit tricky, but tf.RadgedTensor helped with stacking tensors of different shapes, produced by tf.map_fn. It works, but unfortunately, it is very slow, since I need to process each vector individually, in a 3D tensor of shape [None, 543, 3]. Since the example above deals only with vectors, I've tried to exploit tf.map_fn again, in order to map my mean_by_segments per frame axis and then per coordinate axis (yes, one tf.map_fn call another tf.map_fn. Now, it works as expected in autograph mode, but it is dramatically slow. For instance, processing one (sic!) sample takes from 200ms to 2s+ (depending on hardware). Having that, it is impossible ven to train a model using such preprocessing (even not talking about submission).
\nCould you provide any though on this? Thank you in advance.
Hi there! Are there any constrains on the model submission for using supported_ops?
\nIf so, it should be updated on the Submission Requirements
My submission size is les than 2mb and time is under 100ms but still gives an error.
\nThe only different thing I have from other submissions is that Im using this:
converter.target_spec.supported_ops = [\n tf.lite.OpsSet.TFLITE_BUILTINS, \n tf.lite.OpsSet.SELECT_TF_OPS \n]\nme too.
\nI faced the following error
\n\nConverterError: /opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py:561:0: error: 'tf.TensorListReserve' op requires element_shape to be 1D tensor during TF Lite transformation pass
\n
\n :0: note: loc(fused[\"StatefulPartitionedCall:\", \"StatefulPartitionedCall\"]): called from
\n /opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py:561:0: error: failed to legalize operation 'tf.TensorListReserve' that was explicitly marked illegal
\n :0: note: loc(fused[\"StatefulPartitionedCall:\", \"StatefulPartitionedCall\"]): called from
\n :0: error: Lowering tensor list ops is failed. Please consider using Select TF ops and disabling_experimental_lower_tensor_list_opsflag in the TFLite converter object. For example, converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]\\n converter._experimental_lower_tensor_list_ops = False
and tried to prevent this error to add following code.
\n\n\nkeras_model_converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
\n
\n keras_model_converter._experimental_lower_tensor_list_ops = False
It worked fine in kaggle notebook, but my submission failed.
\nUnfortunately, the error don't show which part(operator) is the error, so I can't solve the problem now.
", "rawMarkdown": "me too.\nI faced the following error \n>ConverterError: /opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py:561:0: error: 'tf.TensorListReserve' op requires element_shape to be 1D tensor during TF Lite transformation pass\n:0: note: loc(fused[\"StatefulPartitionedCall:\", \"StatefulPartitionedCall\"]): called from\n/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py:561:0: error: failed to legalize operation 'tf.TensorListReserve' that was explicitly marked illegal\n:0: note: loc(fused[\"StatefulPartitionedCall:\", \"StatefulPartitionedCall\"]): called from\n:0: error: Lowering tensor list ops is failed. Please consider using Select TF ops and disabling `_experimental_lower_tensor_list_ops` flag in the TFLite converter object. For example, converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]\\n converter._experimental_lower_tensor_list_ops = False\n\nand tried to prevent this error to add following code.\n>keras_model_converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]\nkeras_model_converter._experimental_lower_tensor_list_ops = False\n\nIt worked fine in kaggle notebook, but my submission failed.\n\nUnfortunately, the error don't show which part(operator) is the error, so I can't solve the problem now.", "replies": [ { "id": 2195297, "postDate": "2023-03-24T14:59:22.013Z", "content": "I've solve the problem by only using simple operator.
\nBy rewriting function with simple operator step by step and converting, I found the source of problem.
Hi all, I Hope you all are well.
\nMy TFLite model works fine with tflite-runtime version 2.11.0, However, it causes the notebook to crash if I use 2.9.1 instead.
\nThe notebook crashes while initializing the interpreter.
\ninterpreter = tflite.Interpreter(tflite_model_path)
\nwhile loading the interpreter in tflite-runtime version 2.11.0, I get following warnings.
\nAny help would be appreciated. Thanks
", "rawMarkdown": "Hi all, I Hope you all are well. \n\nMy TFLite model works fine with tflite-runtime version 2.11.0, However, it causes the notebook to crash if I use 2.9.1 instead.\n\nThe notebook crashes while initializing the interpreter.\n\ninterpreter = tflite.Interpreter(tflite_model_path)\n\nwhile loading the interpreter in tflite-runtime version 2.11.0, I get following warnings.\n\n- INFO: Created TensorFlow Lite delegate for select TF ops.\n- INFO: TfLiteFlexDelegate delegate: 30 nodes delegated out of 166 nodes with 2 partitions.\n\nAny help would be appreciated. Thanks" }, { "id": 2173799, "postDate": "2023-03-08T16:42:54.613Z", "content": "How do I convert from Pytorch and use tensorflow without running into any bugs ?
", "rawMarkdown": "How do I convert from Pytorch and use tensorflow without running into any bugs ?", "replies": [ { "id": 2198675, "postDate": "2023-03-27T08:26:25.443Z", "content": "I think this answer may be what you're looking for (unless you figured it out already): https://www.kaggle.com/competitions/asl-signs/discussion/390008#2157381
", "rawMarkdown": "I think this answer may be what you're looking for (unless you figured it out already): https://www.kaggle.com/competitions/asl-signs/discussion/390008#2157381 " } ] }, { "id": 2171174, "postDate": "2023-03-06T15:03:59.573Z", "content": "Hi all,
\nI have two questions:
\n(1) several preprocessed datasets are currently in use: if we generate our own preprocessed dataset (e.g. with a different batch size or different preprocessing), do we need to regenerate this from scratch in our final submission? Or do we only need to generate it once and is any dataset we generate automatically coupled to the notebook that generated it? I'm also not sure how (where) to save a self-generated dataset from one notebook in such a way we can use it from a different notebook.
\n(2) W.r.t. the time constraint for TfLite inference: does a submission fail when that is exceeded? And if not: how can we know whether our submission meets this constraint?
\n\nI'm also not sure how (where) to save a self-generated dataset from one notebook in such a way we can use it from a different notebook.
\n
As I understand, you can add your own dataset through upload button in top right of the notebook, and then you can add it in any notebook. You can also set visibility setting to private if you don't want others to see it.
", "rawMarkdown": ">I'm also not sure how (where) to save a self-generated dataset from one notebook in such a way we can use it from a different notebook.\n\nAs I understand, you can add your own dataset through upload button in top right of the notebook, and then you can add it in any notebook. You can also set visibility setting to private if you don't want others to see it. " } ] }, { "id": 2169836, "postDate": "2023-03-05T13:28:47.837Z", "content": "Hi,
\nI'm using TensorFlow to make my model for this competition. I tried enabling mixed precision (using float16 and float32 during the training) to reduce the training time, which works just fine. But when I try to convert my (final) trained model into TensorFlow Lite format for submission, the conversion takes too much time. (I waited about 5 hours before manually stopping the conversion code; the code for conversion just stayed hung.) When I don't enable global mixed precision policy in TensorFlow, the conversion works fine, and it takes about a minute or so.
\nThis is my conversion code:
\n\nconverter = tf.lite.TFLiteConverter.from_keras_model(final_model)\n\ntflite_model = converter.convert()\nmodel_path = \n\n\n (model_path, ) f:\n f.write(tflite_model)\n\n\n! submission. $model_path\nDo you have any workarounds for this problem?
\nBTW, I use TensorFlow 2.10 and 2.11 for training (and conversion).
", "rawMarkdown": "Hi,\n\nI'm using TensorFlow to make my model for this competition. I tried enabling mixed precision (using float16 and float32 during the training) to reduce the training time, which works just fine. But when I try to convert my (final) trained model into TensorFlow Lite format for submission, the conversion takes too much time. (I waited about 5 hours before manually stopping the conversion code; the code for conversion just stayed hung.) When I don't enable global mixed precision policy in TensorFlow, the conversion works fine, and it takes about a minute or so. \n\nThis is my conversion code:\n\n```Python\n\n# Convert the final model to a 'tflite' model\nconverter = tf.lite.TFLiteConverter.from_keras_model(final_model)\n\ntflite_model = converter.convert()\nmodel_path = \"model.tflite\"\n\n# Save the tflite model\nwith open(model_path, 'wb') as f:\n f.write(tflite_model)\n\n# Make the 'submission.zip'\n!zip submission.zip $model_path\n\n```\n\nDo you have any workarounds for this problem?\n\nBTW, I use TensorFlow 2.10 and 2.11 for training (and conversion).\n", "replies": [ { "id": 2169854, "postDate": "2023-03-05T13:52:12.870Z", "content": "Just a simple check, your using the right tflite runtime? I believe from the evaluation page it’s 2.9.1.
\nAn alternative to help with troubleshooting would be to try to make a simple, reproducible notebook.
\nIe
\nIf it doesn’t fail, it’s probably something specific to comparability in one of the layers and potentially the interaction of that layer with mixed precision.
\nYou could also try training in full precision on your original notebook and converting as a simple check to make sure it has something to do with the mixed precision and not a particular op or layer
", "rawMarkdown": "Just a simple check, your using the right tflite runtime? I believe from the evaluation page it’s 2.9.1.\n\nAn alternative to help with troubleshooting would be to try to make a simple, reproducible notebook.\n\nIe \n1. New Notebook\n2. Initialize simple model with mixed precision (literally any model)\n3. Try to convert\n\nIf it doesn’t fail, it’s probably something specific to comparability in one of the layers and potentially the interaction of that layer with mixed precision.\n\nYou could also try training in full precision on your original notebook and converting as a simple check to make sure it has something to do with the mixed precision and not a particular op or layer ", "votes": 3, "replies": [ { "id": 2170558, "postDate": "2023-03-06T05:16:42.490Z", "content": "Great suggestions. Thanks for the help.
", "rawMarkdown": "Great suggestions. Thanks for the help." } ] } ] }, { "id": 2169118, "postDate": "2023-03-04T19:44:49.497Z", "content": "Dear experts,
\nI'm still confused about one thing in TfLite. Tensorflow models always have a 'hidden' first input dimension corresponding to batch size, with 'None' size to account for variable batch size. In the code posted by Lonnie, for the inference model, the batch dimension is 'abused' as a time dimension. When you train a variable time step time series model in Tensorflow, you typically have two None dimensions: one for batch size and one for frames. I've already tried many things, but haven't found a way to get this into TfLight. So is this at all possible and if so: how? Do we need to pass specific arguments to the TfLite model generation? This would be by far the easiest solution since you could then directly compile your trained model instead of having to reconstruct it in the inference model code (this is particularly painful if your model has branches and merges).
\nAs an alternative, I also tried adding a size-1 first dimension inside the inference model (since batch size would always be 1 at inference time), but never managed to get this running: whatever I try, I get errors either at compile time or when trying to run it for inference.
", "rawMarkdown": "Dear experts,\n\nI'm still confused about one thing in TfLite. Tensorflow models always have a 'hidden' first input dimension corresponding to batch size, with 'None' size to account for variable batch size. In the code posted by Lonnie, for the inference model, the batch dimension is 'abused' as a time dimension. When you train a variable time step time series model in Tensorflow, you typically have two None dimensions: one for batch size and one for frames. I've already tried many things, but haven't found a way to get this into TfLight. So is this at all possible and if so: how? Do we need to pass specific arguments to the TfLite model generation? This would be by far the easiest solution since you could then directly compile your trained model instead of having to reconstruct it in the inference model code (this is particularly painful if your model has branches and merges).\n\nAs an alternative, I also tried adding a size-1 first dimension inside the inference model (since batch size would always be 1 at inference time), but never managed to get this running: whatever I try, I get errors either at compile time or when trying to run it for inference. ", "replies": [ { "id": 2169178, "postDate": "2023-03-04T20:26:20.500Z", "content": "I don't have any direct evidence or proof, but my assumption is for inference we will get a single test entry at a time (frames, 543, 3). Never (batch, frames, 543, 3). That would break the contract we were told to expect in the Evaluation instructions.
\nSo your inference model can expect None, 543, 3 and can choose to do whatever tricks desired with the variable dimension. Averaging, padding, and \"resize\"-ing: personally I'm doing all three individually then concatenating together
", "rawMarkdown": "I don't have any direct evidence or proof, but my assumption is for inference we will get a single test entry at a time (frames, 543, 3). Never (batch, frames, 543, 3). That would break the contract we were told to expect in the Evaluation instructions. \n\nSo your inference model can expect None, 543, 3 and can choose to do whatever tricks desired with the variable dimension. Averaging, padding, and \"resize\"-ing: personally I'm doing all three individually then concatenating together" }, { "id": 2170165, "postDate": "2023-03-05T18:20:03.253Z", "content": "Adding a 1-sized batch dimension works for me. For a working example you can refer to my notebook.
", "rawMarkdown": "Adding a 1-sized batch dimension works for me. For a working example you can refer to [my notebook](https://www.kaggle.com/code/aapokossi/submission-for-variable-length-time-series-model).", "votes": 2, "replies": [ { "id": 2170208, "postDate": "2023-03-05T18:54:11.837Z", "content": "Wow, thanks!
", "rawMarkdown": "Wow, thanks!\n" } ] } ] }, { "id": 2168628, "postDate": "2023-03-04T11:39:40.887Z", "content": "Hello, I've been having difficulties submitting my work and I think I finally understood why : I was using the tf.atan function in my work, which isn't handled by TFLite.
\nI tried some workaround (tf.asin/tf.angle for instance) but it seems every one of them is failing when converting the model to TFLite. Do you happen to know of a native tensorflow, TFLite-supported, operation to compute angles ?
Inner products of normalised verctors give you cosines …
", "rawMarkdown": "Inner products of normalised verctors give you cosines ... ", "replies": [ { "id": 2169710, "postDate": "2023-03-05T11:07:52.253Z", "content": "Yes, and after one got the cosine, one need to use an arc-cos function to actually get the angle, or am I wrong ?
", "rawMarkdown": "Yes, and after one got the cosine, one need to use an arc-cos function to actually get the angle, or am I wrong ?" }, { "id": 2170214, "postDate": "2023-03-05T18:54:38.207Z", "content": "Yes, but maybe cosine is enough?
", "rawMarkdown": "Yes, but maybe cosine is enough?\n", "votes": 1 } ] } ] }, { "id": 2168166, "postDate": "2023-03-04T00:14:39.820Z", "content": "Any suggestions how to do a basic np.nanmean on a given axis in TF Lite?
\nI eventually found that this code below doesn't work because ragged tensor ops aren't supported in conversion. I was using it to preserve dimensions, which is obviously necessary (for only a split second) for the 'reduce_mean' to work correctly.
\n face_center = tf.reduce_mean(tf.ragged.boolean_mask(inputs[:, :, :], tf.math.is_finite(inputs[:, :, :])), axis=, keepdims=)\n face_center = face_center.to_tensor(default_value=(), shape=[, , ])\nGot it working. (h/t @dschettler8845 )
\nThe wrapper functions I'm using:
\n ():\n tf.reduce_sum(tf.where(tf.math.is_nan(x), tf.zeros_like(x), x), axis=axis) / tf.reduce_sum(tf.where(tf.math.is_nan(x), tf.zeros_like(x), tf.ones_like(x)), axis=axis)\n\n ():\n d = x - tf_nan_mean(x, axis=axis)\n tf.math.sqrt(tf_nan_mean(d * d, axis=axis))\nNice! Glad it is working.
\nOne suggestion. You may wish to use tf.math.divide_no_nan as opposed to the / operator. It'll handle the case of all zeros or (non-finite) in the denominator. That being said I'm not sure if it's supported in tflite…
():\n tf.math.divide_no_nan(tf.reduce_sum(tf.where(tf.math.is_nan(x), tf.zeros_like(x), x), axis=axis),\n tf.reduce_sum(tf.where(tf.math.is_nan(x), tf.zeros_like(x), tf.ones_like(x)), axis=axis))\n\n ():\n d = x - tf_nan_mean(x, axis=axis)\n tf.math.sqrt(tf_nan_mean(d * d, axis=axis))\nI assume \"SELECT_TF_OPS\" is NOT supported and will fail in scoring?
\nkeras_model_converter.target_spec.supported_ops = [\n tf.lite.OpsSet.TFLITE_BUILTINS, \n tf.lite.OpsSet.SELECT_TF_OPS \n]\nI'm having a lot of trouble with the preprocessing.
\nMy current problem can, hopefully, be simplified as very similar to wanting to resize a variable shape image to a fixed size:
\nSomething like tf.keras.layers.Resizing
Examples that will never work(?)
\nFrom https://www.kaggle.com/code/lonnieqin/isolated-sign-language-recognition-with-dnn
inputs = tf.keras.Input((, ), dtype=tf.float32, name=)\n x = custom_preprocess(inputs)\nThis is symbolic, so won't work with my custom_preprocess function that has special logic based on the number of frames.
\nAlong those lines, is there some way that this code COULD work with tf.keras.layers.Resizing?
-
\nFrom https://www.kaggle.com/code/dschettler8845/gislr-learn-eda-baseline/notebook
\n@tf.function(input_signature=[tf.TensorSpec(shape=[None, 543, 3], dtype=tf.float32, name='inputs')])
\nI think this is similar to the other example? I doubt it can work with my custom function. No idea whether it's possible or how to convert it to use something like tf.keras.layers.Resizing.
Is there some approach that should work? Taking a variable dimension and dynamically converting it to a fixed size dimension using custom logic seems like a very common type of problem.
", "rawMarkdown": "I'm having a lot of trouble with the preprocessing.\n\nMy current problem can, hopefully, be simplified as very similar to wanting to resize a variable shape image to a fixed size:\nSomething like tf.keras.layers.Resizing\n\nExamples that will never work(?)\nFrom https://www.kaggle.com/code/lonnieqin/isolated-sign-language-recognition-with-dnn\n```python\n inputs = tf.keras.Input((543, 3), dtype=tf.float32, name=\"inputs\")\n x = custom_preprocess(inputs)\n```\nThis is symbolic, so won't work with my custom_preprocess function that has special logic based on the number of frames.\nAlong those lines, is there some way that this code COULD work with tf.keras.layers.Resizing?\n\n-\nFrom https://www.kaggle.com/code/dschettler8845/gislr-learn-eda-baseline/notebook\n` @tf.function(input_signature=[tf.TensorSpec(shape=[None, 543, 3], dtype=tf.float32, name='inputs')])`\nI think this is similar to the other example? I doubt it can work with my custom function. No idea whether it's possible or how to convert it to use something like tf.keras.layers.Resizing.\n\nIs there some approach that should work? Taking a variable dimension and dynamically converting it to a fixed size dimension using custom logic seems like a very common type of problem.", "replies": [ { "id": 2167125, "postDate": "2023-03-03T09:22:33.413Z", "content": "Try tf.image.resize, you can find some examples in
\n https://www.tensorflow.org/api_docs/python/tf/image/resize
Yeah, that's a good starting point, at least.
\nIf that works, I guess I'll have to port the underlying resize code directly into my Notebook, and debug/derive what that code does to enable itself to work, and use the learnings to write my own custom logic.
\nOr accept a pre-built resize algorithm and give up on writing my own logic.
", "rawMarkdown": "Yeah, that's a good starting point, at least.\n\nIf that works, I guess I'll have to port the underlying resize code directly into my Notebook, and debug/derive what that code does to enable itself to work, and use the learnings to write my own custom logic.\n\nOr accept a pre-built resize algorithm and give up on writing my own logic." }, { "id": 2167721, "postDate": "2023-03-03T17:09:49.180Z", "content": "At least it works!
\nIt's pretty terrible because it doesn't handle nan, and filling with 0s distorts the resized values… but at least it works.
", "rawMarkdown": "At least it works!\n\nIt's pretty terrible because it doesn't handle nan, and filling with 0s distorts the resized values... but at least it works." }, { "id": 2168048, "postDate": "2023-03-03T20:30:02.710Z", "rawMarkdown": "", "isDeleted": true } ] } ] }, { "id": 2164090, "postDate": "2023-03-01T09:47:41.210Z", "content": "Hi, I am so excited to attend this competition. I have some questions about the concentration from pytorch model to TensorFlow mode.
\nI know we can use onnx to do it. We need to install onnx_tf, which requires internet access. However, as a rule, says, we could n use the internet in the notebook. So how do we use onnx_tf? Is there a way to install it without using the internet?
\n!pip install onnx_tf
I think(?) you can simply use the internet in notebook 1, up to an including creating the tflite model, then use the result in notebook 2 that has internet off and makes the final submission.
There's a couple simple ways discussed here: https://www.kaggle.com/competitions/icecube-neutrinos-in-deep-ice/discussion/382898
My sign language processing research is fundamentally impossible to run with tflite -
\nI first transcribe the sign represented as a pose sequence to SignWriting (autoregressive translation model, not supported in tflite), then I perform classification on the result.
\nI will not be able to participate unless you at the very least allow multiple tflite files, and a python function to, given a model, perform the inference loop.
How can I learn tensorflow in kaggle, Can someone guide me pls?
", "rawMarkdown": "How can I learn tensorflow in kaggle, Can someone guide me pls?", "replies": [ { "id": 2162057, "postDate": "2023-02-28T00:44:49.040Z", "content": "Hi Ranjan, take a look at this guide: https://www.kaggle.com/learn-guide/tensorflow
", "rawMarkdown": "Hi Ranjan, take a look at this guide: https://www.kaggle.com/learn-guide/tensorflow" } ] }, { "id": 2159921, "postDate": "2023-02-26T06:23:00.787Z", "content": "Do you happen to have example notebook/script that can convert LightGBM/XGBoost -> ONNX -> TensorFlow -> TensorFlow Lite? Looks technically feasible with this ONNX toolset. https://github.com/onnx/onnxmltools
", "rawMarkdown": "Do you happen to have example notebook/script that can convert LightGBM/XGBoost -> ONNX -> TensorFlow -> TensorFlow Lite? Looks technically feasible with this ONNX toolset. https://github.com/onnx/onnxmltools " }, { "id": 2158650, "postDate": "2023-02-25T03:28:37.700Z", "content": "What resources would you suggest for beginners to learn TF Lite, given some familiarity with Tensorflow ?
", "rawMarkdown": "What resources would you suggest for beginners to learn TF Lite, given some familiarity with Tensorflow ?", "replies": [ { "id": 2158683, "postDate": "2023-02-25T04:21:30.087Z", "content": "So it really depends on how you learn best and what your current experience level is with ML in general. A few places I'd start would be the official documentation pages and the YouTube channel with a bit of searching for what you want to learn. Hopefully that helps at least get you started :)
", "rawMarkdown": "So it really depends on how you learn best and what your current experience level is with ML in general. A few places I'd start would be the [official documentation pages](https://www.tensorflow.org/lite) and the [YouTube channel](https://www.youtube.com/@TensorFlow) with a bit of searching for what you want to learn. Hopefully that helps at least get you started :)" } ] }, { "id": 2157830, "postDate": "2023-02-24T11:12:49.203Z", "content": "Can we have the dataset in any object detection format- TF or YOLO?
", "rawMarkdown": "Can we have the dataset in any object detection format- TF or YOLO?", "replies": [ { "id": 2158531, "postDate": "2023-02-24T23:38:36.483Z", "content": "We are only going to provide the data in the existing format.
", "rawMarkdown": "We are only going to provide the data in the existing format." } ] }, { "id": 2157564, "postDate": "2023-02-24T05:55:17.933Z", "content": "The competition requires the use TensorFlow. What if the model I come up with is fundamentally different from commonly used machine learning models today? (Specifically, fundamentally different from a neural network.)
", "rawMarkdown": "The competition requires the use TensorFlow. What if the model I come up with is fundamentally different from commonly used machine learning models today? (Specifically, fundamentally different from a neural network.)", "replies": [ { "id": 2158166, "postDate": "2023-02-24T17:05:15.180Z", "content": "You're welcome to submit anything you can make work with TensorFlow Lite.
", "rawMarkdown": "You're welcome to submit anything you can make work with TensorFlow Lite.", "replies": [ { "id": 2158262, "postDate": "2023-02-24T18:13:35.500Z", "content": "Can TensorFlow Lite work with a non-neural network model? I'm happy to try getting things work in TensorFlow but every doc I look, it seems that TensorFlow is only for neural networks.
", "rawMarkdown": "Can TensorFlow Lite work with a non-neural network model? I'm happy to try getting things work in TensorFlow but every doc I look, it seems that TensorFlow is only for neural networks. " }, { "id": 2158604, "postDate": "2023-02-25T02:18:19.607Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2158605, "postDate": "2023-02-25T02:19:45.350Z", "content": "You may find this post on running k-nearest neighbors with Tensorflow Lite helpful: https://ata-tech.medium.com/running-k-nearest-neighbors-on-tensorflow-lite-e3affba4d706
", "rawMarkdown": "You may find this post on running k-nearest neighbors with Tensorflow Lite helpful: https://ata-tech.medium.com/running-k-nearest-neighbors-on-tensorflow-lite-e3affba4d706" } ] } ] }, { "id": 2220628, "postDate": "2023-04-13T14:53:09.157Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2217997, "postDate": "2023-04-11T10:27:15.153Z", "rawMarkdown": "", "votes": -2, "isDeleted": true }, { "id": 2179008, "postDate": "2023-03-12T20:22:37.343Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2167533, "postDate": "2023-03-03T14:50:48.947Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2161923, "postDate": "2023-02-27T20:37:05.853Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2160830, "postDate": "2023-02-27T02:52:07.107Z", "rawMarkdown": "", "votes": 3, "isDeleted": true }, { "id": 2157901, "postDate": "2023-02-24T12:45:06.923Z", "rawMarkdown": "", "votes": 1, "isDeleted": true, "replies": [ { "id": 2158279, "postDate": "2023-02-24T18:36:21.057Z", "rawMarkdown": "", "isDeleted": true } ] } ], "comments": [ { "id": 2160198, "author_name": "Andrew", "author_url": "", "post_date": "2023-02-26T13:13:00.060000", "content": "\n\nYour model must also require less than 40 MB in memory
\n
How do we calculate the in-memory size of a model? What specific checks does the submission evaluation run - so that we can pre-check our models and avoid wasted submissions.
", "votes": 11, "replies": [ { "id": 2169859, "author_name": "Darien Schettler", "author_url": "", "post_date": "2023-03-05T13:58:28.253000", "content": "This is my rule of thumb:
\nCalculate the memory usage of the model by multiplying the number of parameters by the number of bytes required to store each parameter. For example, if your model has 10 million parameters and each parameter requires 4 bytes, the total memory usage would be 40 MB.
\nNote that the actual memory usage of the model may differ from this calculated value due to various factors such as the use of GPU memory, data type of the parameters, and any compression or optimization techniques used.
\nI’d also note that it appears more permissive than the above because I’ve definitely submitted models that are 10M+ that have succeeded.
\nHope that helps!
", "votes": 6, "replies": [] } ] }, { "id": 2162315, "author_name": "Lakshmi Narayana Podagatlapalli", "author_url": "", "post_date": "2023-02-28T05:59:02.477000", "content": "I recently started learning Tensorflow and wish to confirm that my understanding is correct:
\nIs there a list somewhere that tells us what layers are compatible with TFlite? Since we are dealing with sequences of data, I tried using LSTM and GRU, but both don't seem to be compatible with TFlite. It technically runs, but when I convert to TFlite I get this warning:
\nWARNING:absl:Found untraced functions such as _update_step_xla, lstm_cell_1_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_2_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 5). These functions will not be directly callable after loading.
\nThe accuracy when using the TFlite model is much lower than when using regular TF model. Assuming it's because GRU and LSTM are not compatible with TFlite?
", "votes": 3, "replies": [] }, { "id": 2157188, "author_name": "Narendra Kumar Reddy36", "author_url": "", "post_date": "2023-02-23T19:05:00.070000", "content": "As, I am at starting state of learning, I don't known weather it will complete or not. But for me this project seams to be a great one and I will try to learn at least something while working on this.
\nthank you for this amazing project
", "votes": 3, "replies": [] }, { "id": 2158548, "author_name": "Chen Lin", "author_url": "", "post_date": "2023-02-25T00:32:07.537000", "content": "\n\nYour model must also require less than 40 MB in memory and perform inference with less than 100 milliseconds of latency per video.
\n
How should we calculate the inference latency for each video on the validation set? Should we calculate the time between reading data from disk and getting inference results? Because I assume we can't get all the data to be read into RAM or graphics memory
", "votes": 4, "replies": [ { "id": 2160593, "author_name": "Dhananjaya Swain", "author_url": "", "post_date": "2023-02-26T19:46:55.430000", "content": "It talks about the storage and runtime your trained model takes for a given input. Hope this clarifies your query!
", "votes": -3, "replies": [] }, { "id": 2169858, "author_name": "Darien Schettler", "author_url": "", "post_date": "2023-03-05T13:56:41.720000", "content": "For inference timing you can time the inference code that they provide in a loop and time it. If your using a variable time series, I would do the 75% upper quantile of number of frames to mimic a slightly longer than average runtime. That should give you a bit of buffer.
\nFor size, calculate the memory usage of the model by multiplying the number of parameters by the number of bytes required to store each parameter. For example, if your model has 10 million parameters and each parameter requires 4 bytes, the total memory usage would be 40 MB.
\nNote that the actual memory usage of the model may differ from this calculated value due to various factors such as the use of GPU memory, data type of the parameters, and any compression or optimization techniques used.
Hope that helps!
", "votes": 0, "replies": [] } ] }, { "id": 2157243, "author_name": "DrHB", "author_url": "", "post_date": "2023-02-23T20:35:48.100000", "content": "what is the best tools/way to convert Pytorch model to tflite?
", "votes": 1, "replies": [ { "id": 2157381, "author_name": "Mark Sherwood", "author_url": "", "post_date": "2023-02-24T01:03:09.540000", "content": "We see many PyTorch developers who deploy via TensorFlow Lite use a path of PyTorch -> ONNX -> TensorFlow -> TensorFlow Lite. Two community posts demonstrating this path are https://medium.com/geekculture/converting-yolo-v7-to-tensorflow-lite-for-mobile-deployment-ebc1103e8d1e and https://github.com/sayakpaul/Adventures-in-TensorFlow-Lite/blob/master/TUNIT_Conversion_to_TF_Lite.ipynb
", "votes": 9, "replies": [] } ] }, { "id": 2157385, "author_name": "ONODERA", "author_url": "", "post_date": "2023-02-24T01:04:14.190000", "content": "\n\nThis competition requires submissions to be made in the form of TensorFlow Lite models. You are welcome to train your model using the framework of your choice as long as you convert the model checkpoint into the tflite format prior to submission. Please see the evaluation page for details.
\n
What if we did not use TensorFlow Lite models? We will be removed from Leaderboard?
", "votes": 0, "replies": [ { "id": 2157387, "author_name": "Sohier Dane", "author_url": "", "post_date": "2023-02-24T01:07:14.163000", "content": "Your submission will fail without a TF Lite model.
", "votes": 3, "replies": [ { "id": 2216731, "author_name": "Jh Probas", "author_url": "", "post_date": "2023-04-10T09:52:33.367000", "content": "Cool, thank you!
", "votes": 0, "replies": [] } ] } ] }, { "id": 2217824, "author_name": "WeiweiDuan", "author_url": "", "post_date": "2023-04-11T07:39:54.293000", "content": "Thanks for you amazing code, it helps me a lot
", "votes": -2, "replies": [] }, { "id": 2232348, "author_name": "Haoyu Jiang", "author_url": "", "post_date": "2023-04-24T08:31:08.840000", "content": "very good.this code help me a lot
", "votes": 0, "replies": [] }, { "id": 2226249, "author_name": "Stanislav Cherkasov", "author_url": "", "post_date": "2023-04-18T19:17:47.850000", "content": "Hello
\nI receive error message:
error: op neither a custom op nor a flex op\nerror: failed converting: : \nSome ops are supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select \nTF Select ops: Conv2D\nDetails:\n tf.Conv2D(tensor<*xf32>, tensor<2x2x3x32xf32>) -> (tensor<?x?x?x32xf32>) : {data_format = , device = , dilations = [, , , ], explicit_paddings = [], padding = , strides = [, , , ], use_cudnn_on_gpu = true}\nWhy it's happens? According https://www.tensorflow.org/mlir/tfl_ops?hl=ru#tflconv_2d_mlirtflconv2dop conv_2d layers are supported
", "votes": 0, "replies": [] }, { "id": 2218660, "author_name": "Andrew", "author_url": "", "post_date": "2023-04-11T22:27:55.273000", "content": "TFlite models seem to grow in size very quickly for just a few more features in the data. Are there standard ways to shrink the size?
", "votes": 0, "replies": [] }, { "id": 2198405, "author_name": "deoxy", "author_url": "", "post_date": "2023-03-27T01:22:30.350000", "content": "Several people have already mentioned the topic of \"TensorFlow operators\".
\nAs far as I can confirm, the current submission environment does not support TensorFlow operators, I would expect.
\nI checked on the Kaggle notebook and it is possible to support TensorFlow operators in tflite_runtime by importing tensorflow as follows
@@ -1,3 +1,4 @@\n+import tensorflow\n import tflite_runtime.interpreter as tflite\n interpreter = tflite.Interpreter(model_path)\nIt seems that the introduction of TensorFlow operators is not impractical for device implementations.
\nhttps://www.tensorflow.org/lite/guide/ops_select
Is there any possibility of this modification to the submission environment?
\nI would be very happy if TensorFlow operators were introduced, as I think it would make it easier to implement more complex models and increase our chances of achieving greater results.
\nI welcome any comments on implementation issues on the device, concerns about competitiveness, etc.
", "votes": 0, "replies": [ { "id": 2199328, "author_name": "Sohier Dane", "author_url": "", "post_date": "2023-03-27T16:27:59.200000", "content": "We're deliberately excluding TensorFlow operators as the goal of the competition is to generate models that could potentially run on cell phones.
", "votes": 1, "replies": [] } ] }, { "id": 2189855, "author_name": "Mykola", "author_url": "", "post_date": "2023-03-20T20:52:02.770000", "content": "Hello, guys. I faced an issue with tf.data pipeline working in graph mode. I need to edit the tensor somehow, but I cannot do so. I understand that tf.Tensor is immutable and supposed to work in a functional style. So, I am creating tf.Variable from it and that works well in eager execution mode. But tf.data pipeline uses graph mode and makes tf.Variable creation in such a way impossible.
\nHere is a minimalistic code, which reproduce my issue:
\n\nimport tensorflow as tf
\nx = tf.constant([1,2,3.0])
\n
\n ds = tf.data.Dataset.from_tensors(x)def test(x):
\n
\n with tf.init_scope():
\n v = tf.Variable(x)ds_x = ds.map(test)
\n
\n next(iter(ds_x))
Here is an error I get
\n`
\nValueError: in user code:
File \"/tmp/ipykernel_3517/140449158.py\", line 6, in test *\n v = tf.Variable(x)\n\nValueError: Argument `initial_value` (Tensor(\"args_0:0\", shape=(3,), dtype=float32)) could not be lifted out of a `tf.function`. (Tried to create variable with name='None'). To avoid this error, when constructing `tf.Variable`s inside of `tf.function` you can create the `initial_value` tensor in a `tf.init_scope` or pass a callable `initial_value` (e.g., `tf.Variable(lambda : tf.truncated_normal([10, 40]))`). Please file a feature request if this restriction inconveniences you.\n`
\nBest to my understanding, I cannot create a tf.Variable which depends on input tensor x because of static execution graph. But how do I change a part of input tensor x, where the decision of that part is depends on x content?
Looking forward to your advice. Thank you in advance
", "votes": 0, "replies": [ { "id": 2190236, "author_name": "Paul Ruiz", "author_url": "", "post_date": "2023-03-21T06:04:22.103000", "content": "Hi Mykola,
\nThis is the code snippet I was sent for your issue:
\nimport tensorflow as tf\n\nx = tf.constant([1,2,3,4,5,6])\nds = tf.data.Dataset.from_tensor_slices(x)\n\n@tf.function\ndef test(x):\n if x > 2:\n return x**2\n else:\n return x\n\nds_x = ds.map(test)\n\nfor x in iter(ds_x):\n print(x)\nYou can also find a lot of examples for tf.data here: https://www.tensorflow.org/guide/data
", "votes": 1, "replies": [ { "id": 2190448, "author_name": "Mykola", "author_url": "", "post_date": "2023-03-21T08:40:22.010000", "content": "I am not sure, whether this solves my problem, but further investigation gives me the following intuition and almost the solution (I am still in progress now). I would like to ask you whether my thoughts are correct or if am I wrong in so something.
\nThe goal is to calculate the mean of the input vector by segments. The actual segments depend on the content of the input vector.
\n ():\n ...\n segments \nAnd now I am going to iterate over each segments and calculate mean
\n ():\n segments = get_segments(x)\n mean_list = []\n i segments: \n mean_list.append(tf.math.reduce_mean(x[i[]:i[])) \n\n means = tf.stack(mean_list) \nThis works pretty well in eager mode, but fails if I decorate it with @tf.function. That makes sense. Creating a mutable list is a violation of functional programming, thus we could not build a static computational graph. In fact, a list works well, if I create it with a predefined length (like, mean_list = [tf.constant(2), tf.math.reduce_mean(x[a:b]), ...]). I guess TensorFlow graph compiler is smart enough to understand that list in that case is not going to be mutable ever.
Regarding the task of calculating mean over segments currently, I found a possible solution with tf.map_fn. I discover a way to pass multiple arguments to fn using tf.map_fn (via tuples) - vector x and segment i in my case. This way enforces me to have the same shape over the first axis. So, I need to tile an x vector N times, in order to have the same length first axis - [N, None] for x and [N, 2] for segments. Hope, that would resolve my current issue.
Taking the opportunity, I would like to ask you the following questions:
\nAm I right, that all code under tf.function should be purely functional?
Does my solution with tf.map_fn the best one for the task described above?
Doesn't the tf.data pipeline support eager execution mode?
Sorry, I some of the questions are trivial, but I spent a loot of time starting from the first release of TF2.0 on this. Thank you in advance!
The solution with tf.map_fn works, until fn returns tensors of different shapes. I want to apply the following function
():\n x = arg[]\n _ = arg[]\n a = r[]\n b = r[]\n x[a:b]*x[] \n\n ():\n segments = get_segments(x)\n a = tf.tile(x, [(segments)]) \n a = tf.reshape(a, [(segments), -])\n c = tf.map_fn(take_it, (a,segments), fn_output_signature=tf.int64)\n\n means = tf.stack(mean_list) \nLet say x is a tensor [0,1,2,3,4,5,6,7,8,9] and get_segments(x) returns the following
[[0, 3],\n [3, 7],\n [7, 10]]\nI expect to get [0,0, 4,6,8,10, 49,63] as a result, but it is impossible since take_it returns tensors of different shapes. Now I get the following error:
2023-03-21 09:52:22.638552: W tensorflow/core/framework/op_kernel.cc:1830] OP_REQUIRES failed at list_kernels.h:206 : INVALID_ARGUMENT: PartialTensorShape: Incompatible shapes during merge: [2] vs. [4]\nCould anyone give me an idea of how to solve my problem with @tf.function approach?
It was a little bit tricky, but tf.RadgedTensor helped with stacking tensors of different shapes, produced by tf.map_fn. It works, but unfortunately, it is very slow, since I need to process each vector individually, in a 3D tensor of shape [None, 543, 3]. Since the example above deals only with vectors, I've tried to exploit tf.map_fn again, in order to map my mean_by_segments per frame axis and then per coordinate axis (yes, one tf.map_fn call another tf.map_fn. Now, it works as expected in autograph mode, but it is dramatically slow. For instance, processing one (sic!) sample takes from 200ms to 2s+ (depending on hardware). Having that, it is impossible ven to train a model using such preprocessing (even not talking about submission).
\nCould you provide any though on this? Thank you in advance.
Hi there! Are there any constrains on the model submission for using supported_ops?
\nIf so, it should be updated on the Submission Requirements
My submission size is les than 2mb and time is under 100ms but still gives an error.
\nThe only different thing I have from other submissions is that Im using this:
converter.target_spec.supported_ops = [\n tf.lite.OpsSet.TFLITE_BUILTINS, \n tf.lite.OpsSet.SELECT_TF_OPS \n]\nme too.
\nI faced the following error
\n\nConverterError: /opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py:561:0: error: 'tf.TensorListReserve' op requires element_shape to be 1D tensor during TF Lite transformation pass
\n
\n :0: note: loc(fused[\"StatefulPartitionedCall:\", \"StatefulPartitionedCall\"]): called from
\n /opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py:561:0: error: failed to legalize operation 'tf.TensorListReserve' that was explicitly marked illegal
\n :0: note: loc(fused[\"StatefulPartitionedCall:\", \"StatefulPartitionedCall\"]): called from
\n :0: error: Lowering tensor list ops is failed. Please consider using Select TF ops and disabling_experimental_lower_tensor_list_opsflag in the TFLite converter object. For example, converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]\\n converter._experimental_lower_tensor_list_ops = False
and tried to prevent this error to add following code.
\n\n\nkeras_model_converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
\n
\n keras_model_converter._experimental_lower_tensor_list_ops = False
It worked fine in kaggle notebook, but my submission failed.
\nUnfortunately, the error don't show which part(operator) is the error, so I can't solve the problem now.
", "votes": 0, "replies": [ { "id": 2195297, "author_name": "Aurora_blue", "author_url": "", "post_date": "2023-03-24T14:59:22.013000", "content": "I've solve the problem by only using simple operator.
\nBy rewriting function with simple operator step by step and converting, I found the source of problem.
Hi all, I Hope you all are well.
\nMy TFLite model works fine with tflite-runtime version 2.11.0, However, it causes the notebook to crash if I use 2.9.1 instead.
\nThe notebook crashes while initializing the interpreter.
\ninterpreter = tflite.Interpreter(tflite_model_path)
\nwhile loading the interpreter in tflite-runtime version 2.11.0, I get following warnings.
\nAny help would be appreciated. Thanks
", "votes": 0, "replies": [] }, { "id": 2173799, "author_name": "steven hinojosa", "author_url": "", "post_date": "2023-03-08T16:42:54.613000", "content": "How do I convert from Pytorch and use tensorflow without running into any bugs ?
", "votes": 0, "replies": [ { "id": 2198675, "author_name": "Abhinav Naikawadi", "author_url": "", "post_date": "2023-03-27T08:26:25.443000", "content": "I think this answer may be what you're looking for (unless you figured it out already): https://www.kaggle.com/competitions/asl-signs/discussion/390008#2157381
", "votes": 0, "replies": [] } ] }, { "id": 2171174, "author_name": "Wondering Alice", "author_url": "", "post_date": "2023-03-06T15:03:59.573000", "content": "Hi all,
\nI have two questions:
\n(1) several preprocessed datasets are currently in use: if we generate our own preprocessed dataset (e.g. with a different batch size or different preprocessing), do we need to regenerate this from scratch in our final submission? Or do we only need to generate it once and is any dataset we generate automatically coupled to the notebook that generated it? I'm also not sure how (where) to save a self-generated dataset from one notebook in such a way we can use it from a different notebook.
\n(2) W.r.t. the time constraint for TfLite inference: does a submission fail when that is exceeded? And if not: how can we know whether our submission meets this constraint?
\n\nI'm also not sure how (where) to save a self-generated dataset from one notebook in such a way we can use it from a different notebook.
\n
As I understand, you can add your own dataset through upload button in top right of the notebook, and then you can add it in any notebook. You can also set visibility setting to private if you don't want others to see it.
", "votes": 0, "replies": [] } ] }, { "id": 2169836, "author_name": "Hassan Abedi", "author_url": "", "post_date": "2023-03-05T13:28:47.837000", "content": "Hi,
\nI'm using TensorFlow to make my model for this competition. I tried enabling mixed precision (using float16 and float32 during the training) to reduce the training time, which works just fine. But when I try to convert my (final) trained model into TensorFlow Lite format for submission, the conversion takes too much time. (I waited about 5 hours before manually stopping the conversion code; the code for conversion just stayed hung.) When I don't enable global mixed precision policy in TensorFlow, the conversion works fine, and it takes about a minute or so.
\nThis is my conversion code:
\n\nconverter = tf.lite.TFLiteConverter.from_keras_model(final_model)\n\ntflite_model = converter.convert()\nmodel_path = \n\n\n (model_path, ) f:\n f.write(tflite_model)\n\n\n! submission. $model_path\nDo you have any workarounds for this problem?
\nBTW, I use TensorFlow 2.10 and 2.11 for training (and conversion).
", "votes": 0, "replies": [ { "id": 2169854, "author_name": "Darien Schettler", "author_url": "", "post_date": "2023-03-05T13:52:12.870000", "content": "Just a simple check, your using the right tflite runtime? I believe from the evaluation page it’s 2.9.1.
\nAn alternative to help with troubleshooting would be to try to make a simple, reproducible notebook.
\nIe
\nIf it doesn’t fail, it’s probably something specific to comparability in one of the layers and potentially the interaction of that layer with mixed precision.
\nYou could also try training in full precision on your original notebook and converting as a simple check to make sure it has something to do with the mixed precision and not a particular op or layer
", "votes": 3, "replies": [ { "id": 2170558, "author_name": "Hassan Abedi", "author_url": "", "post_date": "2023-03-06T05:16:42.490000", "content": "Great suggestions. Thanks for the help.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2169118, "author_name": "Wondering Alice", "author_url": "", "post_date": "2023-03-04T19:44:49.497000", "content": "Dear experts,
\nI'm still confused about one thing in TfLite. Tensorflow models always have a 'hidden' first input dimension corresponding to batch size, with 'None' size to account for variable batch size. In the code posted by Lonnie, for the inference model, the batch dimension is 'abused' as a time dimension. When you train a variable time step time series model in Tensorflow, you typically have two None dimensions: one for batch size and one for frames. I've already tried many things, but haven't found a way to get this into TfLight. So is this at all possible and if so: how? Do we need to pass specific arguments to the TfLite model generation? This would be by far the easiest solution since you could then directly compile your trained model instead of having to reconstruct it in the inference model code (this is particularly painful if your model has branches and merges).
\nAs an alternative, I also tried adding a size-1 first dimension inside the inference model (since batch size would always be 1 at inference time), but never managed to get this running: whatever I try, I get errors either at compile time or when trying to run it for inference.
", "votes": 0, "replies": [ { "id": 2169178, "author_name": "Robert Hatch", "author_url": "", "post_date": "2023-03-04T20:26:20.500000", "content": "I don't have any direct evidence or proof, but my assumption is for inference we will get a single test entry at a time (frames, 543, 3). Never (batch, frames, 543, 3). That would break the contract we were told to expect in the Evaluation instructions.
\nSo your inference model can expect None, 543, 3 and can choose to do whatever tricks desired with the variable dimension. Averaging, padding, and \"resize\"-ing: personally I'm doing all three individually then concatenating together
", "votes": 0, "replies": [] }, { "id": 2170165, "author_name": "aapo kossi", "author_url": "", "post_date": "2023-03-05T18:20:03.253000", "content": "Adding a 1-sized batch dimension works for me. For a working example you can refer to my notebook.
", "votes": 2, "replies": [ { "id": 2170208, "author_name": "Wondering Alice", "author_url": "", "post_date": "2023-03-05T18:54:11.837000", "content": "Wow, thanks!
", "votes": 0, "replies": [] } ] } ] }, { "id": 2168628, "author_name": "Augustin Combes", "author_url": "", "post_date": "2023-03-04T11:39:40.887000", "content": "Hello, I've been having difficulties submitting my work and I think I finally understood why : I was using the tf.atan function in my work, which isn't handled by TFLite.
\nI tried some workaround (tf.asin/tf.angle for instance) but it seems every one of them is failing when converting the model to TFLite. Do you happen to know of a native tensorflow, TFLite-supported, operation to compute angles ?
Inner products of normalised verctors give you cosines …
", "votes": 0, "replies": [ { "id": 2169710, "author_name": "Augustin Combes", "author_url": "", "post_date": "2023-03-05T11:07:52.253000", "content": "Yes, and after one got the cosine, one need to use an arc-cos function to actually get the angle, or am I wrong ?
", "votes": 0, "replies": [] }, { "id": 2170214, "author_name": "Wondering Alice", "author_url": "", "post_date": "2023-03-05T18:54:38.207000", "content": "Yes, but maybe cosine is enough?
", "votes": 1, "replies": [] } ] } ] }, { "id": 2168166, "author_name": "Robert Hatch", "author_url": "", "post_date": "2023-03-04T00:14:39.820000", "content": "Any suggestions how to do a basic np.nanmean on a given axis in TF Lite?
\nI eventually found that this code below doesn't work because ragged tensor ops aren't supported in conversion. I was using it to preserve dimensions, which is obviously necessary (for only a split second) for the 'reduce_mean' to work correctly.
\n face_center = tf.reduce_mean(tf.ragged.boolean_mask(inputs[:, :, :], tf.math.is_finite(inputs[:, :, :])), axis=, keepdims=)\n face_center = face_center.to_tensor(default_value=(), shape=[, , ])\nGot it working. (h/t @dschettler8845 )
\nThe wrapper functions I'm using:
\n ():\n tf.reduce_sum(tf.where(tf.math.is_nan(x), tf.zeros_like(x), x), axis=axis) / tf.reduce_sum(tf.where(tf.math.is_nan(x), tf.zeros_like(x), tf.ones_like(x)), axis=axis)\n\n ():\n d = x - tf_nan_mean(x, axis=axis)\n tf.math.sqrt(tf_nan_mean(d * d, axis=axis))\nNice! Glad it is working.
\nOne suggestion. You may wish to use tf.math.divide_no_nan as opposed to the / operator. It'll handle the case of all zeros or (non-finite) in the denominator. That being said I'm not sure if it's supported in tflite…
():\n tf.math.divide_no_nan(tf.reduce_sum(tf.where(tf.math.is_nan(x), tf.zeros_like(x), x), axis=axis),\n tf.reduce_sum(tf.where(tf.math.is_nan(x), tf.zeros_like(x), tf.ones_like(x)), axis=axis))\n\n ():\n d = x - tf_nan_mean(x, axis=axis)\n tf.math.sqrt(tf_nan_mean(d * d, axis=axis))\nI assume \"SELECT_TF_OPS\" is NOT supported and will fail in scoring?
\nkeras_model_converter.target_spec.supported_ops = [\n tf.lite.OpsSet.TFLITE_BUILTINS, \n tf.lite.OpsSet.SELECT_TF_OPS \n]\nI'm having a lot of trouble with the preprocessing.
\nMy current problem can, hopefully, be simplified as very similar to wanting to resize a variable shape image to a fixed size:
\nSomething like tf.keras.layers.Resizing
Examples that will never work(?)
\nFrom https://www.kaggle.com/code/lonnieqin/isolated-sign-language-recognition-with-dnn
inputs = tf.keras.Input((, ), dtype=tf.float32, name=)\n x = custom_preprocess(inputs)\nThis is symbolic, so won't work with my custom_preprocess function that has special logic based on the number of frames.
\nAlong those lines, is there some way that this code COULD work with tf.keras.layers.Resizing?
-
\nFrom https://www.kaggle.com/code/dschettler8845/gislr-learn-eda-baseline/notebook
\n@tf.function(input_signature=[tf.TensorSpec(shape=[None, 543, 3], dtype=tf.float32, name='inputs')])
\nI think this is similar to the other example? I doubt it can work with my custom function. No idea whether it's possible or how to convert it to use something like tf.keras.layers.Resizing.
Is there some approach that should work? Taking a variable dimension and dynamically converting it to a fixed size dimension using custom logic seems like a very common type of problem.
", "votes": 0, "replies": [ { "id": 2167125, "author_name": "Meikintom", "author_url": "", "post_date": "2023-03-03T09:22:33.413000", "content": "Try tf.image.resize, you can find some examples in
\n https://www.tensorflow.org/api_docs/python/tf/image/resize
Yeah, that's a good starting point, at least.
\nIf that works, I guess I'll have to port the underlying resize code directly into my Notebook, and debug/derive what that code does to enable itself to work, and use the learnings to write my own custom logic.
\nOr accept a pre-built resize algorithm and give up on writing my own logic.
", "votes": 0, "replies": [] }, { "id": 2167721, "author_name": "Robert Hatch", "author_url": "", "post_date": "2023-03-03T17:09:49.180000", "content": "At least it works!
\nIt's pretty terrible because it doesn't handle nan, and filling with 0s distorts the resized values… but at least it works.
", "votes": 0, "replies": [] }, { "id": 2168048, "author_name": "", "author_url": "", "post_date": "2023-03-03T20:30:02.710000", "content": "", "votes": 0, "replies": [] } ] } ] }, { "id": 2164090, "author_name": "Meikintom", "author_url": "", "post_date": "2023-03-01T09:47:41.210000", "content": "Hi, I am so excited to attend this competition. I have some questions about the concentration from pytorch model to TensorFlow mode.
\nI know we can use onnx to do it. We need to install onnx_tf, which requires internet access. However, as a rule, says, we could n use the internet in the notebook. So how do we use onnx_tf? Is there a way to install it without using the internet?
\n!pip install onnx_tf
I think(?) you can simply use the internet in notebook 1, up to an including creating the tflite model, then use the result in notebook 2 that has internet off and makes the final submission.
There's a couple simple ways discussed here: https://www.kaggle.com/competitions/icecube-neutrinos-in-deep-ice/discussion/382898
My sign language processing research is fundamentally impossible to run with tflite -
\nI first transcribe the sign represented as a pose sequence to SignWriting (autoregressive translation model, not supported in tflite), then I perform classification on the result.
\nI will not be able to participate unless you at the very least allow multiple tflite files, and a python function to, given a model, perform the inference loop.
How can I learn tensorflow in kaggle, Can someone guide me pls?
", "votes": 0, "replies": [ { "id": 2162057, "author_name": "Glenn Cameron", "author_url": "", "post_date": "2023-02-28T00:44:49.040000", "content": "Hi Ranjan, take a look at this guide: https://www.kaggle.com/learn-guide/tensorflow
", "votes": 0, "replies": [] } ] }, { "id": 2159921, "author_name": "Tim Sereno", "author_url": "", "post_date": "2023-02-26T06:23:00.787000", "content": "Do you happen to have example notebook/script that can convert LightGBM/XGBoost -> ONNX -> TensorFlow -> TensorFlow Lite? Looks technically feasible with this ONNX toolset. https://github.com/onnx/onnxmltools
", "votes": 0, "replies": [] }, { "id": 2158650, "author_name": "Dietzsche Nostoevsky", "author_url": "", "post_date": "2023-02-25T03:28:37.700000", "content": "What resources would you suggest for beginners to learn TF Lite, given some familiarity with Tensorflow ?
", "votes": 0, "replies": [ { "id": 2158683, "author_name": "Paul Ruiz", "author_url": "", "post_date": "2023-02-25T04:21:30.087000", "content": "So it really depends on how you learn best and what your current experience level is with ML in general. A few places I'd start would be the official documentation pages and the YouTube channel with a bit of searching for what you want to learn. Hopefully that helps at least get you started :)
", "votes": 0, "replies": [] } ] }, { "id": 2157830, "author_name": "Prateek Gupta", "author_url": "", "post_date": "2023-02-24T11:12:49.203000", "content": "Can we have the dataset in any object detection format- TF or YOLO?
", "votes": 0, "replies": [ { "id": 2158531, "author_name": "Sohier Dane", "author_url": "", "post_date": "2023-02-24T23:38:36.483000", "content": "We are only going to provide the data in the existing format.
", "votes": 0, "replies": [] } ] }, { "id": 2157564, "author_name": "", "author_url": "", "post_date": "2023-02-24T05:55:17.933000", "content": "", "votes": 0, "replies": [ { "id": 2158166, "author_name": "", "author_url": "", "post_date": "2023-02-24T17:05:15.180000", "content": "", "votes": 0, "replies": [ { "id": 2158262, "author_name": "", "author_url": "", "post_date": "2023-02-24T18:13:35.500000", "content": "", "votes": 0, "replies": [] }, { "id": 2158604, "author_name": "", "author_url": "", "post_date": "2023-02-25T02:18:19.607000", "content": "", "votes": 0, "replies": [] }, { "id": 2158605, "author_name": "", "author_url": "", "post_date": "2023-02-25T02:19:45.350000", "content": "", "votes": 0, "replies": [] } ] } ] }, { "id": 2220628, "author_name": "", "author_url": "", "post_date": "2023-04-13T14:53:09.157000", "content": "", "votes": 0, "replies": [] }, { "id": 2217997, "author_name": "", "author_url": "", "post_date": "2023-04-11T10:27:15.153000", "content": "", "votes": -2, "replies": [] }, { "id": 2179008, "author_name": "", "author_url": "", "post_date": "2023-03-12T20:22:37.343000", "content": "", "votes": 0, "replies": [] }, { "id": 2167533, "author_name": "", "author_url": "", "post_date": "2023-03-03T14:50:48.947000", "content": "", "votes": 0, "replies": [] }, { "id": 2161923, "author_name": "", "author_url": "", "post_date": "2023-02-27T20:37:05.853000", "content": "", "votes": 0, "replies": [] }, { "id": 2160830, "author_name": "", "author_url": "", "post_date": "2023-02-27T02:52:07.107000", "content": "", "votes": 3, "replies": [] }, { "id": 2157901, "author_name": "", "author_url": "", "post_date": "2023-02-24T12:45:06.923000", "content": "", "votes": 1, "replies": [ { "id": 2158279, "author_name": "", "author_url": "", "post_date": "2023-02-24T18:36:21.057000", "content": "", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "2157047": "Hey all! I'm Paul, a developer advocate with the TensorFlow team. I wanted to make a discussion thread here for all TensorFlow Lite related questions so we can follow along in a centralized place and get the correct people around for sharing their knowledge :)", "2160198": "> Your model must also require less than 40 MB in memory\n\nHow do we calculate the in-memory size of a model? What specific checks does the submission evaluation run - so that we can pre-check our models and avoid wasted submissions.", "2162315": "I recently started learning Tensorflow and wish to confirm that my understanding is correct:\n- TFlite is not for building models but for using them in practice.\n- So they can be built in Tensorflow as usual.\n- But tensorflow models by themselves are not tflite compatible.\n- So we use tf.lite.TFLiteConverter to make it tflite compatible.(source: [https://www.tensorflow.org/lite/guide/inference](https://www.tensorflow.org/lite/guide/inference))\nPlease let me know if this is correct or correct where required.\nAlso, please suggest any other methods we can use.", "2170436": "Is there a list somewhere that tells us what layers are compatible with TFlite? Since we are dealing with sequences of data, I tried using LSTM and GRU, but both don't seem to be compatible with TFlite. It technically runs, but when I convert to TFlite I get this warning: \n\nWARNING:absl:Found untraced functions such as _update_step_xla, lstm_cell_1_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_2_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 5). These functions will not be directly callable after loading.\n\nThe accuracy when using the TFlite model is much lower than when using regular TF model. Assuming it's because GRU and LSTM are not compatible with TFlite?", "2157188": "As, I am at starting state of learning, I don't known weather it will complete or not. But for me this project seams to be a great one and I will try to learn at least something while working on this.\n\n\nthank you for this amazing project\n", "2158548": ">Your model must also require less than 40 MB in memory and perform inference with less than 100 milliseconds of latency per video. \n\nHow should we calculate the inference latency for each video on the validation set? Should we calculate the time between reading data from disk and getting inference results? Because I assume we can't get all the data to be read into RAM or graphics memory", "2157243": "what is the best tools/way to convert Pytorch model to tflite?", "2157385": "> This competition requires submissions to be made in the form of TensorFlow Lite models. You are welcome to train your model using the framework of your choice as long as you convert the model checkpoint into the tflite format prior to submission. Please see the evaluation page for details.\n\nWhat if we did not use TensorFlow Lite models? We will be removed from Leaderboard?", "2217824": "Thanks for you amazing code, it helps me a lot", "2232348": "very good.this code help me a lot", "2226249": "Hello\nI receive error message:\n\n```python\nerror: 'tf.Conv2D' op is neither a custom op nor a flex op\nerror: failed while converting: 'main': \nSome ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select \nTF Select ops: Conv2D\nDetails:\n tf.Conv2D(tensor<*xf32>, tensor<2x2x3x32xf32>) -> (tensor) : {data_format = \"NHWC\", device = \"\", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = \"SAME\", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}\n```\nWhy it's happens? According https://www.tensorflow.org/mlir/tfl_ops?hl=ru#tflconv_2d_mlirtflconv2dop conv_2d layers are supported", "2218660": "TFlite models seem to grow in size very quickly for just a few more features in the data. Are there standard ways to shrink the size? ", "2198405": "Several people have already mentioned the topic of \"TensorFlow operators\".\nAs far as I can confirm, the current submission environment does not support TensorFlow operators, I would expect.\nI checked on the Kaggle notebook and it is possible to support TensorFlow operators in tflite_runtime by importing tensorflow as follows\n\n```\n@@ -1,3 +1,4 @@\n+import tensorflow\n import tflite_runtime.interpreter as tflite\n interpreter = tflite.Interpreter(model_path)\n```\n\nIt seems that the introduction of TensorFlow operators is not impractical for device implementations.\nhttps://www.tensorflow.org/lite/guide/ops_select\n\nIs there any possibility of this modification to the submission environment?\n\nI would be very happy if TensorFlow operators were introduced, as I think it would make it easier to implement more complex models and increase our chances of achieving greater results.\n\nI welcome any comments on implementation issues on the device, concerns about competitiveness, etc.\n", "2189855": "Hello, guys. I faced an issue with tf.data pipeline working in graph mode. I need to edit the tensor somehow, but I cannot do so. I understand that tf.Tensor is immutable and supposed to work in a functional style. So, I am creating tf.Variable from it and that works well in eager execution mode. But tf.data pipeline uses graph mode and makes tf.Variable creation in such a way impossible. \nHere is a minimalistic code, which reproduce my issue:\n\n>import tensorflow as tf\n>\n>x = tf.constant([1,2,3.0])\n>ds = tf.data.Dataset.from_tensors(x)\n>\n>def test(x):\n> with tf.init_scope():\n> v = tf.Variable(x)\n>\n>ds_x = ds.map(test)\n>next(iter(ds_x))\n\nHere is an error I get\n\n`\nValueError: in user code:\n\n File \"/tmp/ipykernel_3517/140449158.py\", line 6, in test *\n v = tf.Variable(x)\n\n ValueError: Argument `initial_value` (Tensor(\"args_0:0\", shape=(3,), dtype=float32)) could not be lifted out of a `tf.function`. (Tried to create variable with name='None'). To avoid this error, when constructing `tf.Variable`s inside of `tf.function` you can create the `initial_value` tensor in a `tf.init_scope` or pass a callable `initial_value` (e.g., `tf.Variable(lambda : tf.truncated_normal([10, 40]))`). Please file a feature request if this restriction inconveniences you.\n`\n\nBest to my understanding, I cannot create a tf.Variable which depends on input tensor `x` because of static execution graph. But how do I change a part of input tensor `x`, where the decision of that part is depends on `x` content?\n\nLooking forward to your advice. Thank you in advance", "2185203": "Hi there! Are there any constrains on the model submission for using `supported_ops`?\n**If so, it should be updated on the Submission Requirements**\n\nMy submission size is les than 2mb and time is under 100ms but still gives an error.\nThe only different thing I have from other submissions is that Im using this:\n```python\nconverter.target_spec.supported_ops = [\n tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.\n tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.\n]\n```", "2173968": "Hi all, I Hope you all are well. \n\nMy TFLite model works fine with tflite-runtime version 2.11.0, However, it causes the notebook to crash if I use 2.9.1 instead.\n\nThe notebook crashes while initializing the interpreter.\n\ninterpreter = tflite.Interpreter(tflite_model_path)\n\nwhile loading the interpreter in tflite-runtime version 2.11.0, I get following warnings.\n\n- INFO: Created TensorFlow Lite delegate for select TF ops.\n- INFO: TfLiteFlexDelegate delegate: 30 nodes delegated out of 166 nodes with 2 partitions.\n\nAny help would be appreciated. Thanks", "2173799": "How do I convert from Pytorch and use tensorflow without running into any bugs ?", "2171174": "Hi all,\n\nI have two questions:\n\n(1) several preprocessed datasets are currently in use: if we generate our own preprocessed dataset (e.g. with a different batch size or different preprocessing), do we need to regenerate this from scratch in our final submission? Or do we only need to generate it once and is any dataset we generate automatically coupled to the notebook that generated it? I'm also not sure how (where) to save a self-generated dataset from one notebook in such a way we can use it from a different notebook.\n(2) W.r.t. the time constraint for TfLite inference: does a submission fail when that is exceeded? And if not: how can we know whether our submission meets this constraint?", "2169836": "Hi,\n\nI'm using TensorFlow to make my model for this competition. I tried enabling mixed precision (using float16 and float32 during the training) to reduce the training time, which works just fine. But when I try to convert my (final) trained model into TensorFlow Lite format for submission, the conversion takes too much time. (I waited about 5 hours before manually stopping the conversion code; the code for conversion just stayed hung.) When I don't enable global mixed precision policy in TensorFlow, the conversion works fine, and it takes about a minute or so. \n\nThis is my conversion code:\n\n```Python\n\n# Convert the final model to a 'tflite' model\nconverter = tf.lite.TFLiteConverter.from_keras_model(final_model)\n\ntflite_model = converter.convert()\nmodel_path = \"model.tflite\"\n\n# Save the tflite model\nwith open(model_path, 'wb') as f:\n f.write(tflite_model)\n\n# Make the 'submission.zip'\n!zip submission.zip $model_path\n\n```\n\nDo you have any workarounds for this problem?\n\nBTW, I use TensorFlow 2.10 and 2.11 for training (and conversion).\n", "2169118": "Dear experts,\n\nI'm still confused about one thing in TfLite. Tensorflow models always have a 'hidden' first input dimension corresponding to batch size, with 'None' size to account for variable batch size. In the code posted by Lonnie, for the inference model, the batch dimension is 'abused' as a time dimension. When you train a variable time step time series model in Tensorflow, you typically have two None dimensions: one for batch size and one for frames. I've already tried many things, but haven't found a way to get this into TfLight. So is this at all possible and if so: how? Do we need to pass specific arguments to the TfLite model generation? This would be by far the easiest solution since you could then directly compile your trained model instead of having to reconstruct it in the inference model code (this is particularly painful if your model has branches and merges).\n\nAs an alternative, I also tried adding a size-1 first dimension inside the inference model (since batch size would always be 1 at inference time), but never managed to get this running: whatever I try, I get errors either at compile time or when trying to run it for inference. ", "2168628": "Hello, I've been having difficulties submitting my work and I think I finally understood why : I was using the tf.atan function in my work, which isn't handled by TFLite. \nI tried some workaround (tf.asin/tf.angle for instance) but it seems every one of them is failing when converting the model to TFLite. Do you happen to know of a native tensorflow, TFLite-supported, operation to compute angles ?", "2168166": "Any suggestions how to do a basic np.nanmean on a given axis in TF Lite?\n\nI eventually found that this code below doesn't work because ragged tensor ops aren't supported in conversion. I was using it to preserve dimensions, which is obviously necessary (for only a split second) for the 'reduce_mean' to work correctly.\n\n```python\n face_center = tf.reduce_mean(tf.ragged.boolean_mask(inputs[:, 0:468, :], tf.math.is_finite(inputs[:, 0:468, :])), axis=1, keepdims=True)\n face_center = face_center.to_tensor(default_value=float('nan'), shape=[None, 1, 3])\n```", "2168049": "I assume \"SELECT_TF_OPS\" is NOT supported and will fail in scoring?\n\n```python\nkeras_model_converter.target_spec.supported_ops = [\n tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.\n tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.\n]\n```", "2166680": "I'm having a lot of trouble with the preprocessing.\n\nMy current problem can, hopefully, be simplified as very similar to wanting to resize a variable shape image to a fixed size:\nSomething like tf.keras.layers.Resizing\n\nExamples that will never work(?)\nFrom https://www.kaggle.com/code/lonnieqin/isolated-sign-language-recognition-with-dnn\n```python\n inputs = tf.keras.Input((543, 3), dtype=tf.float32, name=\"inputs\")\n x = custom_preprocess(inputs)\n```\nThis is symbolic, so won't work with my custom_preprocess function that has special logic based on the number of frames.\nAlong those lines, is there some way that this code COULD work with tf.keras.layers.Resizing?\n\n-\nFrom https://www.kaggle.com/code/dschettler8845/gislr-learn-eda-baseline/notebook\n` @tf.function(input_signature=[tf.TensorSpec(shape=[None, 543, 3], dtype=tf.float32, name='inputs')])`\nI think this is similar to the other example? I doubt it can work with my custom function. No idea whether it's possible or how to convert it to use something like tf.keras.layers.Resizing.\n\nIs there some approach that should work? Taking a variable dimension and dynamically converting it to a fixed size dimension using custom logic seems like a very common type of problem.", "2164090": "Hi, I am so excited to attend this competition. I have some questions about the concentration from pytorch model to TensorFlow mode.\nI know we can use onnx to do it. We need to install onnx_tf, which requires internet access. However, as a rule, says, we could n use the internet in the notebook. So how do we use onnx_tf? Is there a way to install it without using the internet?\n!pip install onnx_tf", "2161974": "My sign language processing research is fundamentally impossible to run with tflite - \nI first transcribe the sign represented as a pose sequence to SignWriting (autoregressive translation model, not supported in tflite), then I perform classification on the result. \nI will not be able to participate unless you at the very least allow *multiple* tflite files, and a python function to, given a model, perform the inference loop.", "2160341": "How can I learn tensorflow in kaggle, Can someone guide me pls?", "2159921": "Do you happen to have example notebook/script that can convert LightGBM/XGBoost -> ONNX -> TensorFlow -> TensorFlow Lite? Looks technically feasible with this ONNX toolset. https://github.com/onnx/onnxmltools ", "2158650": "What resources would you suggest for beginners to learn TF Lite, given some familiarity with Tensorflow ?", "2157830": "Can we have the dataset in any object detection format- TF or YOLO?", "2157564": "The competition requires the use TensorFlow. What if the model I come up with is fundamentally different from commonly used machine learning models today? (Specifically, fundamentally different from a neural network.)", "2220628": "", "2217997": "", "2179008": "", "2167533": "", "2161923": "", "2160830": "", "2157901": "" } }