Official tensorflow serving tutorial is not working in colab

TensorFlow General Discussion
,
1

The official tensorflow serving simple tutorial (link - Train and serve a TensorFlow model with TensorFlow Serving  |  TFX) is not working in colab when I use the run in colab option.

The part of code which is throwing error is this:

headers = {"content-type": "application/json"}
json_response = requests.post('http://localhost:8501/v1/models/fashion_model:predict', data=data, headers=headers)
predictions = json.loads(json_response.text)['predictions']

The exact error it is giving is this:

ConnectionError: HTTPConnectionPool(host=‘localhost’, port=8501): Max retries exceeded with url: /v1/models/fashion_model:predict (Caused by NewConnectionError(‘<urllib3.connection.HTTPConnection object at 0x7fc6c23768b0>: Failed to establish a new connection: [Errno 111] Connection refused’))

Please tell me what should I do? Also, please fix this in the official tutorial notebook.

serving #tfserving #tensorflow_serving tf-serving #tensorflow-serving #tf_serving #tfs Tensorflow #tf

3

@Nitish_Pandey_NSL,

Welcome to the Tensorflow Forum!

As a workaround, downgrade tensorflow-model-server from 2.11.0 to 2.8.0.

In Install TensorFlow Serving section of the notebook, please uncomment !{SUDO_IF_NEEDED} apt-get install tensorflow-model-server line and add below lines.

Code to install tensorflow-model-server 2.8.0:

!wget 'http://storage.googleapis.com/tensorflow-serving-apt/pool/tensorflow-model-server-2.8.0/t/tensorflow-model-server/tensorflow-model-server_2.8.0_all.deb'
!dpkg -i tensorflow-model-server_2.8.0_all.deb
!pip3 install tensorflow-serving-api==2.8.0

Please refer to the working gist. Thank you!

5

If you found a fix, please submit a PR to the TFX repo to fix this for everyone.

6

Thank you Mark.

Submitted `Rest_Simple` tutorial is not working in Colab by chunduriv · Pull Request #5602 · tensorflow/tfx · GitHub

1 Like
7

Thank you very much @chunduriv for providing a workaround.