Are there any tips/workarounds to reduce the time encoder-decoder model takes to make prediction ? I had a encoder-decoder model in tf 1. I recently updated it to tf 2. However, now when requests are sent in bulk, the model performs slowly as compared to tf 1. Is there a way to reduce the time taken by encoder-decoder model to make prediction ?
Welcome to the TensorFlow Forum!
Please share minimal reproducible code to replicate the error and understand the issue better.
You can use Tensorflow GPU to accelerate the model performance or can use tf.data input pipelines for the dataset loading in the model. Thank you.