Last Friday (2022-11-11), I successfully compiled Tensorflow from source, based on the instructions I found in Tensorflow.
Before compiling under Fedora 36, I executed these commands:
> sudo dnf install python3-pip python3-devel python3-virtualenv
> pip install -U --user pip numpy wheel packaging requests opt_einsum
> pip install -U --user keras_preprocessing --no-deps
Bazel was installed by downloading and executing the installer (https://github.com/bazelbuild/bazel/releases/download/5.3.2/bazel-5.3.2-installer-linux-x86_64.sh). Before executing ./configure
, I had to change the number in ~/Git/tensorflow/.bazelversion
to 5.3.2
.
./configure was executed with default options whereafter
bazel build //tensorflow/tools/pip_package:build_pip_package
resulted in a build_pip_package
that could be installed (after installing wheel
).
Yesterday, I upgraded to Fedora 37. There were no problems but my tensorflow-installation was gone . I executed the same commands as above. But now, after compiling approximately 10700 files, the process aborted.
[bengbers@fedora tensorflow]$ bazel build //tensorflow/tools/pip_package:build_pip_package
WARNING: Ignoring JAVA_HOME, because it must point to a JDK, not a JRE.
Starting local Bazel server and connecting to it...
INFO: Options provided by the client:
Inherited 'common' options: --isatty=1 --terminal_columns=238
INFO: Reading rc options for 'build' from /home/bengbers/Git/tensorflow/.bazelrc:
Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /home/bengbers/Git/tensorflow/.bazelrc:
'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false
INFO: Reading rc options for 'build' from /home/bengbers/Git/tensorflow/.tf_configure.bazelrc:
'build' options: --action_env PYTHON_BIN_PATH=/usr/bin/python3 --action_env PYTHON_LIB_PATH=/usr/lib/python3.11/site-packages --python_path=/usr/bin/python3
INFO: Reading rc options for 'build' from /home/bengbers/Git/tensorflow/.bazelrc:
'build' options: --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/ir,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_jitrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/common,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils
INFO: Found applicable config definition build:short_logs in file /home/bengbers/Git/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /home/bengbers/Git/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:linux in file /home/bengbers/Git/tensorflow/.bazelrc: --host_copt=-w --copt=-Wno-all --copt=-Wno-extra --copt=-Wno-deprecated --copt=-Wno-deprecated-declarations --copt=-Wno-ignored-attributes --copt=-Wno-unknown-warning --copt=-Wno-array-parameter --copt=-Wno-stringop-overflow --copt=-Wno-array-bounds --copt=-Wunused-result --copt=-Werror=unused-result --copt=-Wswitch --copt=-Werror=switch --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --config=dynamic_kernels --distinct_host_configuration=false --experimental_guard_against_concurrent_changes
INFO: Found applicable config definition build:dynamic_kernels in file /home/bengbers/Git/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/f027b6182beb76f0f73c63208c54d96ce33d0d12.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
INFO: Analyzed target //tensorflow/tools/pip_package:build_pip_package (566 packages loaded, 31655 targets configured).
INFO: Found 1 target...
ERROR: /home/bengbers/.cache/bazel/_bazel_bengbers/d5e1513df739af824db2721edbf62765/external/com_google_protobuf/BUILD:812:10: Compiling python/google/protobuf/pyext/descriptor.cc failed: (Exit 1): gcc failed: error executing command /usr/lib64/ccache/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 71 arguments skipped)
external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc: In function 'bool google::protobuf::python::_CalledFromGeneratedFile(int)':
external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:108:18: error: invalid use of incomplete type 'PyFrameObject' {aka 'struct _frame'}
108 | frame = frame->f_back;
| ^~
In file included from bazel-out/k8-opt/bin/external/local_config_python/python_include/Python.h:42,
from external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:33:
bazel-out/k8-opt/bin/external/local_config_python/python_include/pytypedefs.h:22:16: note: forward declaration of 'PyFrameObject' {aka 'struct _frame'}
22 | typedef struct _frame PyFrameObject;
| ^~~~~~
external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:114:12: error: invalid use of incomplete type 'PyFrameObject' {aka 'struct _frame'}
114 | if (frame->f_code->co_filename == NULL) {
| ^~
bazel-out/k8-opt/bin/external/local_config_python/python_include/pytypedefs.h:22:16: note: forward declaration of 'PyFrameObject' {aka 'struct _frame'}
22 | typedef struct _frame PyFrameObject;
| ^~~~~~
In file included from bazel-out/k8-opt/bin/external/local_config_python/python_include/Python.h:44:
external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:119:37: error: invalid use of incomplete type 'PyFrameObject' {aka 'struct _frame'}
119 | if (PyString_AsStringAndSize(frame->f_code->co_filename,
| ^~
external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:119:7: note: in expansion of macro 'PyString_AsStringAndSize'
119 | if (PyString_AsStringAndSize(frame->f_code->co_filename,
| ^~~~~~~~~~~~~~~~~~~~~~~~
bazel-out/k8-opt/bin/external/local_config_python/python_include/pytypedefs.h:22:16: note: forward declaration of 'PyFrameObject' {aka 'struct _frame'}
22 | typedef struct _frame PyFrameObject;
| ^~~~~~
external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:119:37: error: invalid use of incomplete type 'PyFrameObject' {aka 'struct _frame'}
119 | if (PyString_AsStringAndSize(frame->f_code->co_filename,
| ^~
external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:60:56: note: in definition of macro 'PyString_AsStringAndSize'
60 | PyUnicode_AsUTF8AndSize(ob, (sizep)))) == NULL \
| ^~
bazel-out/k8-opt/bin/external/local_config_python/python_include/pytypedefs.h:22:16: note: forward declaration of 'PyFrameObject' {aka 'struct _frame'}
22 | typedef struct _frame PyFrameObject;
| ^~~~~~
external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:119:37: error: invalid use of incomplete type 'PyFrameObject' {aka 'struct _frame'}
119 | if (PyString_AsStringAndSize(frame->f_code->co_filename,
| ^~
external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:63:50: note: in definition of macro 'PyString_AsStringAndSize'
63 | : PyBytes_AsStringAndSize(ob, (charpp), (sizep)))
| ^~
bazel-out/k8-opt/bin/external/local_config_python/python_include/pytypedefs.h:22:16: note: forward declaration of 'PyFrameObject' {aka 'struct _frame'}
22 | typedef struct _frame PyFrameObject;
| ^~~~~~
external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:140:12: error: invalid use of incomplete type 'PyFrameObject' {aka 'struct _frame'}
140 | if (frame->f_globals != frame->f_locals) {
| ^~
bazel-out/k8-opt/bin/external/local_config_python/python_include/pytypedefs.h:22:16: note: forward declaration of 'PyFrameObject' {aka 'struct _frame'}
22 | typedef struct _frame PyFrameObject;
| ^~~~~~
external/com_google_protobuf/python/google/protobuf/pyext/descriptor.cc:140:32: error: invalid use of incomplete type 'PyFrameObject' {aka 'struct _frame'}
140 | if (frame->f_globals != frame->f_locals) {
| ^~
bazel-out/k8-opt/bin/external/local_config_python/python_include/pytypedefs.h:22:16: note: forward declaration of 'PyFrameObject' {aka 'struct _frame'}
22 | typedef struct _frame PyFrameObject;
| ^~~~~~
At global scope:
cc1plus: note: unrecognized command-line option '-Wno-unknown-warning' may have been intended to silence earlier diagnostics
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 18082.085s, Critical Path: 465.17s
INFO: 7008 processes: 220 internal, 6788 local.
FAILED: Build did NOT complete successfully
I now managed to install tensorflow with pip but I would prefer to install from source.
How can I build tensorflow under Fedora 37?
Ben