Saving models in tf 2.16.1

There is an issue when my model is being saved using tf.saved_model.save in tf 2.16.1. The error is that, there is a type missmatch for a certain feature (my_text_feature) is expecting string and getting float).

Input ‘input_bytes’ of ‘DecodePaddedRaw’ Op has type float32 that does not match expected type of string.

Arguments received by BytesSequenceToEncodingBiLSTM.call():
  • inputs=tf.Tensor(shape=(32, 1), dtype=float32)
  • training=None

I made sure my data types are correct, plus the model training and evaluation completed successfully. This error happens only when trying to save the model.

I did some tracing for the error and I found the following. The call to tf.saved_model.save would invoke my model twice. The first call to it the feature value for my_text_feature is correct (string) which is then transformed to floats (after applying BiLSTM embedding), which is correct.
however the second call passes float to my_text_feature instead of string which crashed in the embedding phase.

I did some tracing and found that this function _build_meta_graph_impl in tensorflow.python.saved_model.save.py gets called when tf.saved_model.save is called. this function _build_meta_graph_impl make the call twice.
#first call
signatures, wrapped_functions, defaults = (signature_serialization.canonicalize_signatures(signatures))
#second call
signature_serialization.validate_augmented_graph_view(augmented_graph_view)

Can someone help why the second call is problematic and why this issue did not rise in older tf version (specifically 2.9)

1 Like

Hi @M_Zahran, As tf2.16.1 by default comes with keras3 could you please try to save the model using model.save() with .keras extension, for example model.save(‘model.keras’) if you still face the error please provide the stand alone code to reproduce the issue and also let us know which keras version you are using. Thank You.