Issue with GPU Delegate (Stuck on 2.3.0)

Oleksandr_Grument · April 18, 2022, 1:13am

Hello everyone. I’m using in android project tensorflow for 2 years and stuck on GpuDelegate V2.3.0
After this versions(2.4.0, 2.5.0, 2.6.0, 2.7.0, 2.8.0) all versions containt some error on GpuDelegate initizialization.

Failed to apply delegate: TfLiteGpuDelegate Init: MUL: Expected a 3D tensor of shape HxWxC or a 4D tensor of shape 1xHxWxC but got 170x8

here is the link for issue, that last almost 2 years

github.com/tensorflow/tensorflow

Failed to apply delegate: TfLiteGpuDelegate Init: MUL: Expected a 3D tensor of shape HxWxC or a 4D tensor of shape 1xHxWxC but got 98x8 (Android)

opened 11:40AM - 18 Dec 20 UTC

closed 07:54PM - 15 Jun 22 UTC

OleksandrGrument

stat:awaiting tensorflower type:bug comp:lite TF 2.4

**System information** - OS Platform and Distribution: Android 9, 10 , 11 - Mo…bile device : Pixel 3a, Nokia 6.1 - TensorFlow installed from: https://bintray.com/google/tensorflow/tensorflow-lite-gpu https://bintray.com/google/tensorflow/tensorflow-lite - TensorFlow version : 2.4.0 **Describe the current behavior** I upgraded tensorflow-lite and tensorflow-lite-gpu from 2.3.0 to 2.4.0 And getting this error on initialization interpeteur ` java.lang.IllegalArgumentException: Internal error: Failed to apply delegate: TfLiteGpuDelegate Init: MUL: Expected a 3D tensor of shape HxWxC or a 4D tensor of shape 1xHxWxC but got 98x8 TfLiteGpuDelegate Prepare: delegate is not initialized Node number 329 (TfLiteGpuDelegateV2) failed to prepare.` ` val tfliteOptions = Interpreter .Options() .setNumThreads(THREADS_COUNT) .addDelegate(GpuDelegate()) Interpreter(loadModelFile(context), tfliteOptions)` The problem is somewhere in tensorflow-lite-gpu 2.4.0 If i'm using such configuration with older version all works well implementation("org.tensorflow:tensorflow-lite:2.4.0") implementation("org.tensorflow:tensorflow-lite-gpu:2.3.0")

Maybe that I’m doing something wrong, and something is possible to do with this.
i’m using trained with magenta model converted from pb to tflite

Maybe this is issue of conversion of old pb to tflite, and there is some feature how to finetune for gpu delegate.

Will be appreciate for any help or advice. Thanks

Oleksandr_Grument · April 18, 2022, 7:17am

Here is Tflite Model GpuDelegateBugReproduce/app/src/main/assets/model.tflite at master · OleksandrGrument/GpuDelegateBugReproduce · GitHub

Sample android project how to reproduce GitHub - OleksandrGrument/GpuDelegateBugReproduce

George · April 18, 2022, 3:45pm

Hi,

Pinging @khanhlvg to shed some light on if it is possible.

khanhlvg · April 20, 2022, 7:31am

Could you try to convert the model from pb file to TFLite using the latest version of TFLiteConverter? The issue could possibly come from version mismatch between TFLiteConverter and the Interpreter you use to run the model.

Oleksandr_Grument · April 20, 2022, 7:47am

I tested on all tf converter versions 2.3.0-2.8.0
I have found the solution yesterday, the issue is with tf.math.multiply(weights, var), I replaced with tf.linalg.matmul(weights ,var, transpose_a=True). Looks like something changed in gpu delegate and it can’t multiply now 2d arrays with common mul operation, like in 2.3.0 version.