I am using tensorflow serving for my model server and want to make it so that when i make a change to my model configs, the models are reloaded, but currently tensorflow serving only seems to reload models and implement the config changes when they are completely unloaded first. I am wondering if there is a way to force a model reload using something like rpc HandleReloadConfigRequest or if this rpc call is also designed to not “unnecessarily” reload models.
Welcome to the Tensorflow Forum,
There are two ways to reload the Model Server configuration:
- By setting the
--model_config_file_poll_wait_secondsflag to instruct the server to periodically check for a new config file at--model_config_filefilepath.
docker run -t --rm -p 8501:8501 \
-v "$(pwd)/models/:/models/" tensorflow/serving \
--model_config_file=/models/models.config \
--model_config_file_poll_wait_seconds=60
- By issuing HandleReloadConfigRequest RPC calls to the server and supplying a new Model Server config programmatically.
Please note that each time the server loads the new config file, it will act to realize the content of the new specified config and only the new specified config. This means if model A was present in the first config file, which is replaced with a file that contains only model B, the server will load model B and unload model A.
Thank you!