YOLOv5 onnx-tf: can't use `summary`, `predict` after conversion, tf.keras.models.load_model returns _UserObject

TensorFlow General Discussion
1

Hi, guys :slight_smile:
I was trying to convert custom trained yolov5s model to tensorflow model for only predict.
First, converting yolov5s to onnx model was successful by running export.py, and to tensorflow representation too.
Pb folder created, and there are assets(but just empty folder), variables folder and saved_model.pb file.

With them, I used tf.keras.models.load_model, the type of model was _UserObject.
I cannot use summary, predict because the model is _UserObject.

Is there something wrong with it?

[version]
tensorflow-gpu : 2.3.0
onnx : 1.8.1
onnx-tf : 1.8.0

2 Likes
2

Hi @human Can you share some of the code here, so the community can try to debug it with you? So the issue is, based on your description, is that, after the onnx-tf conversion, you can’t use either tf.keras.Model’s summary or predict.

2 Likes
3

[yolov5s → onnx] runnned in google colab

1
!python /content/drive/MyDrive/yolov5/models/export.py --train --weights /content/drive/MyDrive/yolov5/runs/yolov5_results6/weights/best.pt --img 512 --batch 1
cs

[onnx → tensorflow representation → pb folder]

1
2
3
4
5
6
import onnx
from onnx_tf.backend import prepare
 
onnx_model = onnx.load(r'test\convert_pt_to_tf\weights\best2.onnx')  # load onnx model
tf_rep = prepare(onnx_model)  # prepare tf representation
tf_rep.export_graph(r'test\convert_pt_to_tf\best2_pb')  # export the model
cs

[pb folder → tensorflow model]

1
2
3
import tensorflow as tf
 
pb_model = tf.keras.models.load_model(r'best2_pb')
cs
2 Likes
4

Okay, so there are two flavors of saved_model: “vanilla” and “keras”.

Vanilla only has the basic TensorFlow constructs (functions, variables).

The “keras” flavored ones also have all the metadata required to rebuild the keras objects.

It can’t automatically uncompile the low-level representation up into a higher-level keras representation.

tf.keras.models.load_model should be printing a warning that there’s no keras metadata available in that saved_model. IIRC, future versions of tensorflow will fail if you use tf.keras.models.load_model on a saved_model that doesn’t have it.

Use tf.saved_model.load. It will return the same _UserObect. Inspect that that to find your functions. does it have a .signatures attirbute?

4 Likes
5

Fortunately, .signatures returns something

1
2
pb_model = tf.keras.models.load_model(r'best2_pb')
pb_model.signatures
cs

[return]
_SignatureMap({'serving_default': <ConcreteFunction signature_wrapper(images) at 0x1E30B312910>})

1 Like
6

I am facing the same issue after converting onnx model to tensorflow, and trying to get model.summary(), it doesn’t work:
import onnx
from onnx_tf.backend import prepare

Load the ONNX model

onnx_model = onnx.load(“model_1.onnx”)
tf_rep = prepare(onnx_model) # prepare tf representation
tf_rep.export_graph(“./model_1”) # export the model as .pb

Load tf model and use

from tensorflow import keras
keras_model = keras.models.load_model(‘model_1’)
keras_model.summary()

Output: ---------------------------------------------------------------------------

  AttributeError                            Traceback (most recent call last)
  <ipython-input-298-81ed83d75a0f> in <module>
  ----> 1 keras_model.summary()

  AttributeError: '_UserObject' object has no attribute 'summary'
8

Facing the same issue as above. The github issue has not yet been resolved so I have little hope for this working.
https://github.com/tensorflow/models/issues/8990

10

Did you solve this issue ? I am also facing the same issue while working with a .pb model which is converted from a onnx model

12

Hi @Sruthi_K @human @Anurag_T @Ostyk, please check out the explanation by @markdaoust here: '_UserObject' object has no attribute 'summary' · Issue #8990 · tensorflow/models · GitHub

…
There are many problems and solutions on this thread.

Issues should be specific and reproducible.

Many of the issues stem from the fact that there are two flavors of saved-model: “generic” and “keras”.

Model.save() will save a “keras” flavored model. tf.keras.models.load_model will load it back as a keras model. Standard keras methods like summary and perdict will all work.

tf.saved_model.save OTOH will save a generic model. tf.saved_model.load will load it back either type, as a generic _UserObject model.

If you load a generic _UserObject model It will only have two sets of methods available:

  1. The tf.function methods that were attached to the model before you saved it
  2. The .signatures that were attached for serving.

See Using the SavedModel format  |  TensorFlow Core for more details.
…

1 Like
13

Actually reading more of the replies here I see that my answer there is somewhat incomplete.

@Anurag_T

keras_model = keras.models.load_model(‘model_1’)
keras_model.summary()

AttributeError: '_UserObject' object has no attribute 'summary'

It looks like both keras.models.load_model will also load either flavor, and just fall-back to the non-keras _UserObject if the saved_model directory doesn’t contain the keras metadata required to load it as a keras.Model