CPython 3.14 Free-Threaded TensorFlow Support — Upstream PR Series and Validation

TensorFlow
1

CPython 3.14 Free-Threaded TensorFlow Support — Upstream PR Series and Validation

Hi everyone,

I have been working on making TensorFlow and parts of its Python runtime ecosystem compatible with CPython 3.14 free-threaded mode (PEP 703).

The work started as an audit of TensorFlow native Python extensions for places that still implicitly relied on the GIL for synchronization, Python object lifetime, callback safety, or mutable native state.

As the work progressed, some issues also turned out to belong to adjacent upstream projects such as LiteRT and gRPC, so those fixes are being submitted to their respective repositories rather than being kept inside TensorFlow.

This post summarizes the current upstream work, the validation performed so far, and the areas where feedback from maintainers and contributors would be useful.

Background

With a traditional CPython build, many native extensions can accidentally rely on the GIL as a process-wide synchronization mechanism.

Under free-threaded Python, that assumption no longer holds.

For TensorFlow, this means native bindings need to be reviewed for issues such as:

  • mutable C++ state accessed concurrently from Python
  • borrowed Python references whose lifetime may change concurrently
  • Python callbacks invoked while native locks are held
  • global initialization paths that previously relied on GIL serialization
  • races between object shutdown/destruction and concurrent operations
  • native modules that need to explicitly declare free-threaded compatibility

The goal of this work is not simply to add py::mod_gil_not_used() to extensions.

The underlying code paths also need to be safe when Python threads can execute them concurrently.

TensorFlow Core Runtime

The main TensorFlow contribution is:

tensorflow/tensorflow#125652

Make core Python runtime bindings safe for free-threaded Python

This PR consolidates the remaining core-runtime work from the free-threading audit.

The affected areas include:

  • TensorFlow session bindings
  • eager / TFE runtime bindings
  • event and debug-event writers
  • Python object ownership and reference handling
  • file I/O bindings
  • stacktrace handling
  • SavedModel bindings
  • quantization-related bindings
  • Python utility bindings
  • nest
  • tf_stack
  • fast_module_type
  • supporting regression coverage

Synchronization and lifetime changes

A recurring pattern in the audit was code that was safe only because the traditional GIL implicitly serialized access.

For free-threaded execution, synchronization has to become explicit.

Examples include:

  • protecting mutable native state with appropriate mutexes
  • taking stable or owned references to Python objects when concurrent mutation is possible
  • avoiding Python callback execution while holding native locks that could create re-entrancy or deadlock problems
  • protecting shared registration/cache state
  • synchronizing event writer and file I/O state
  • auditing weak-reference and callback lifetime handling in eager runtime paths

Python object ownership

Another important area is borrowed-reference behavior.

With free-threaded execution, code cannot always assume that an object retrieved from a Python container will remain alive while another thread may mutate that container.

Some bindings therefore need to use owned references or stable snapshots before continuing native processing.

This affected several runtime and utility paths that interact with:

  • dictionaries
  • lists
  • tuples
  • registered Python objects
  • callback collections
  • stack-trace mapping/filtering state

Validation

The complete working free-threaded TensorFlow snapshot was tested with a freshly built CPython 3.14 free-threaded TensorFlow wheel.

One important validation was checking whether a normal TensorFlow import caused CPython to fall back to GIL-enabled execution.

Observed result:

GIL_BEFORE_IMPORT: False
TF_VERSION: 2.22.0-dev0+selfbuilt
GIL_AFTER_IMPORT: False
TOTAL_WARNING_COUNT: 0
GIL_WARNING_COUNT: 0
NATURAL_LOADED_TF_NATIVE_MODULES: 54
NATURAL_IMPORT_GIL: False -> False
NATURAL_IMPORT_NO_GIL=PASS

In that environment:

  • the GIL was disabled before importing TensorFlow
  • the GIL remained disabled after import tensorflow as tf
  • 54 TensorFlow native modules were reached through the natural import path
  • no GIL-declaration warnings were observed

The work also included subsystem-specific build, runtime, regression, and stress testing during the audit.

This does not mean that every possible TensorFlow execution path has been proven free-threading safe yet, but it provides a useful baseline for continuing the upstream review.

LiteRT / TensorFlow Lite Runtime

Part of the original work covered the TensorFlow Lite Python runtime.

That work was initially submitted as:

tensorflow/tensorflow#125644

During review, it was clarified that TFLite / LiteRT development and runtime bindings have moved to the dedicated google-ai-edge/LiteRT repository.

The runtime changes were therefore migrated to:

google-ai-edge/LiteRT#9687

Make LiteRT Python runtime bindings safe for free-threaded Python

The LiteRT PR is intentionally limited to runtime changes and does not include TensorFlow fuzzing tests.

The LiteRT work includes:

  • marking analyzer, interpreter, and metrics pybind modules with py::mod_gil_not_used()
  • adding per-interpreter synchronization
  • protecting mutable interpreter operations
  • avoiding waits on the interpreter mutex while holding the legacy GIL on normal CPython builds
  • making NumPy C API initialization thread-safe
  • serializing metrics exporter access
  • adding the corresponding Abseil synchronization dependencies

One example of a race exposed by free-threaded execution was NumPy initialization.

Previously:

void ImportNumpy() { import_array1(); }

Multiple free-threaded interpreter creation calls could reach this initialization concurrently.

The migrated implementation uses one-time initialization:

void ImportNumpy() {
  static absl::once_flag flag;
  absl::call_once(flag, []() {
    import_array1();
  });
}

This ensures the NumPy C API initialization path is executed once in a thread-safe manner.

gRPC Python

The free-threading work also exposed an independent concurrency issue in gRPC Python.

That fix is being handled upstream in:

grpc/grpc#43278

The issue involved a race between:

  • retrieving or creating a registered call handle
  • closing the channel concurrently

The fix synchronizes access through the channel state condition so the registered call handle cannot race with channel shutdown.

The relevant validation included:

  • CPython 3.14t wheel build/install
  • checking that importing gRPC does not enable the GIL
  • channel-close race stress testing
  • ChannelCloseTest
  • type smoke tests
  • repeated concurrency stress runs

I mention this here because it illustrates an important aspect of the TensorFlow free-threading effort: some failures observed while validating TensorFlow originate in lower-level or adjacent Python runtime dependencies and need to be fixed in the appropriate upstream project.

Repository Boundaries

One principle I am trying to follow throughout this work is to keep fixes in the project that actually owns the affected runtime.

So far this has resulted in:

  • TensorFlow core runtime fixes → tensorflow/tensorflow
  • TFLite / LiteRT runtime fixes → google-ai-edge/LiteRT
  • gRPC Python runtime race fixes → grpc/grpc

This makes the work easier to review and avoids introducing cross-project fixes into TensorFlow simply because TensorFlow happened to expose the problem during testing.

What I Would Appreciate Feedback On

I would especially appreciate feedback from TensorFlow maintainers and contributors on:

  1. Whether there are additional TensorFlow native modules that should be prioritized for free-threading audit coverage.

  2. Whether there are preferred upstream tests for detecting accidental GIL fallback in TensorFlow native extensions.

  3. Whether the synchronization patterns used in the core-runtime PR align with TensorFlow’s preferred locking and callback-lifetime conventions.

  4. Whether there are existing stress or concurrency suites that would be useful to run specifically under CPython 3.14t.

  5. How maintainers would prefer the remaining free-threading work to be divided across PRs as CPython free-threaded support evolves.

Current Status

The effort is still ongoing.

The important milestone so far is that a working TensorFlow build has been able to execute a natural import under CPython 3.14 free-threaded mode without enabling the GIL, while a number of concrete synchronization and lifetime issues have already been identified and moved upstream to the repositories that own them.

Relevant PRs:

Feedback, additional test suggestions, and review from people familiar with TensorFlow’s native Python runtime would be very welcome.

Thanks!