When will `tfx` support Python 3.9, 3.10 and 3.11?

TensorFlow General Discussion
1

Just wondering when/if tfx will be upgraded to support the newer versions of Python?

Python 3.9, for example, was released on October 5th, 2020.

Thanks!

3

@Robert_Crowe who might be able to help answer

1 Like
4

@Jiayi_Zhao

I believe that TFX 1.7.1 supports Python 3.7 and 3.8 currently. 3.9 will be supported in the next release, coming soon.

1 Like
6

Hi, I just saw that there was a release yesterday, but it doesn’t appear that 3.9 support was part of it. Do you have an ETA for the subsequent release when 3.9 will be supported? Thank you!

1 Like
8

Python 3.9 is supported in the current 1.8.0 release:

10

Thank you! Looking forward to 3.10 support :slight_smile:

12

We have hit an issue upgrading Python from 3.7 to 3.9 with TFX 1.9.1. We compile KFP DAG using this import:

from tfx.orchestration.kubeflow import kubeflow_dag_runner

Given the Python binding (pyext) has not changed since 2016, we guess that there is some difference between these CPython binaries:

_message.cpython-37m-<platform>.so
_message.cpython-39m-<platform>.so

The error is:

Traceback (most recent call last):
  File "/shared/compile.py", line 3, in <module>
    from kfp_compiler.compiler import main
  File "/shared/kfp_compiler/compiler.py", line 7, in <module>
    from tfx.orchestration.kubeflow import kubeflow_dag_runner
  File "/usr/local/lib/python3.9/site-packages/tfx/orchestration/kubeflow/kubeflow_dag_runner.py", line 27, in <module>
    from tfx.dsl.compiler import compiler as tfx_compiler
  File "/usr/local/lib/python3.9/site-packages/tfx/dsl/compiler/compiler.py", line 31, in <module>
    from tfx.dsl.input_resolution.ops import ops
  File "/usr/local/lib/python3.9/site-packages/tfx/dsl/input_resolution/ops/ops.py", line 22, in <module>
    from tfx.dsl.input_resolution.strategies import conditional_strategy
  File "/usr/local/lib/python3.9/site-packages/tfx/dsl/input_resolution/strategies/conditional_strategy.py", line 19, in <module>
    from tfx.dsl.compiler import placeholder_utils
  File "/usr/local/lib/python3.9/site-packages/tfx/dsl/compiler/placeholder_utils.py", line 33, in <module>
    from google.protobuf.pyext import _message
ImportError: cannot import name '_message' from 'google.protobuf.pyext' (/shared/google/protobuf/pyext/__init__.py)

Is this a known issue?

Thanks

14

What version of KFP are you using?

15

We recently updated to KFP 1.7.1. One reason for moving to TFX 1.9.1 is metadata incompatibility between TFX versions and KFP versions.

17

@Jiayi_Zhao

TFX 1.9.1 requires KFP >= 1.8.5 < 2

19

Hi Robert, thx for the reply. What problems might we encounter using KFP 1.7.1? TFX 1.9.1 training pipelines are ‘working’ with 1.7.1.

21

@Jiayi_Zhao @jiyongjung0

I don’t know myself what the problems might be, but the dependency was probably upgraded for a reason.

23

KFP had a conflict in the dependencies with the latest TF release. You can check the related commit.

25

The current version of tfx (1.13.0) supports Python >=3.8, <3.10 .
tfx · PyPI

26

will tfx (1.5.0) support Python 3.10?, I am planning to upgrade to Pthon 3.10

28

The latest release (last september!) added Python 3.10 support: Releases · tensorflow/tfx · GitHub

1 Like