Flutter pytorch model to Tflite

An_Nguy_n · September 15, 2021, 10:17am

I use ONNX to convert Pytorch to tflite and use flutter with package ‘tflite_flutter: ^0.9.0’ to deploy model. But the tflite model run on Colab output differently from flutter (Colab is the correct one). On Colab, image must be transpose to [1,3,256,256] to be feed on the model. But I don’t know how to transpose Image shape on Flutter and feed the image with model shape [1,3,256,256].
After I run ‘interpreter.run(tensorImage.buffer, probabilityBuffer.buffer)’, The output is different from the output from Colab. Output shape from flutter is 1D array which length is [196608].
How to feed image to model shape [1,3,256,256] on flutter.

Bhack · September 15, 2021, 11:20am

Have you tried with

github.com/onnx/onnx-tensorflow

ONNX->"TensorFlow Lite micro" fundamentally broken because of transpose operations

opened 09:39AM - 12 Feb 21 UTC

maartenvds

**Problem** In my usecase, I need to port a model (only containing conv, relu… and maxpool layers) from PyTorch to TensorFlow Lite micro, using the following conversions: PyTorch->ONNX->TensorFLow->Tensorflow Lite micro. The microcontroller kernels of TFLite however only support NHWC format and there is no support for a transpose layer (see supported layers [here](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/micro/all_ops_resolver.cc)). In CPU mode, onnx-tf inserts transpose operations before and after every conv layer so the conv layers can work with NHWC format. However, the rest of the network is kept in NCHW format. What I need is a network with only NHWC tensors for all layers without any transpose operations (also input/output layers should have NHWC format). It would be nice if onnx-tf had a mode where you can completely convert a model that is only using NCHW format to a model that is only using NHWC format instead of creating hybrids as is done by the current implementation. I know this is currently not supported and there is probably not an easy solution for this. But is want to indicate that a conversion from ONNX->TFLite micro is currently fundamentally broken for any given model because of the transpose ops. We could also fix the problem at the TFLite micro side, by implementing a transpose op, but that is a more wasteful solution given the unnecessary overhead of transpose operations on an already very constrained platform. Fixing the issue in TensorFlow by rewriting the graph seems a bit too complex to do and error prone to future changes. I therefore kindly ask to consider implementing an additional mode in the converter if possible. I'm willing to contribute if necessary. **Python, ONNX, ONNX-TF, Tensorflow version** - Python version: 3.6.5 - ONNX version: 1.8.0 - ONNX-TF version: 1.7.0 (master branch, commit: db092105ceebe076610a1b27c1fc1553978c17cd) - Tensorflow version: 2.3.1

lgusm · September 15, 2021, 3:47pm

I don’t have a good answer for the question but out of curiosity, what does this model do?

Topic		Replies	Views
Keras to Tflite using NCHW format General Discussion tflite , help_request	1	956	August 31, 2024
Performance drop with float tflite conversion General Discussion models , tflite	2	611	August 25, 2023
TFLM model with TFLite General Discussion models , tflite	1	739	May 14, 2024
Onnx-tf -> com.microsoft::FusedMatMul operator not implemented General Discussion tflite , pytorch	1	492	December 21, 2023
Converting PyTorch(efficientNet) model to TFLite using ONNX (GPU Binding) General Discussion tflite	1	605	July 11, 2024

Flutter pytorch model to Tflite

Related topics