# Multi compute streams for GPU

**URL:** <https://discuss.ai.google.dev/t/multi-compute-streams-for-gpu/110288>\
**Category:** General Discussion\
**Tags:** gpu, cuda\
**Created:** [November 25, 2025, 8:17am UTC](https://discuss.ai.google.dev/t/multi-compute-streams-for-gpu/110288 "2025-11-25T08:17:57Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Eyal\_Hirsch](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/eyal_hirsch/32/9360_2.png) [@Eyal\_Hirsch](https://discuss.ai.google.dev/u/Eyal_Hirsch)\
**Post date:** [November 25, 2025, 8:17am UTC](https://discuss.ai.google.dev/t/multi-compute-streams-for-gpu/110288/1 "2025-11-25T08:17:57Z")

</div>

Hi,

I’m using TF 2.19.1 with CUDA. Is there a support for multiple StreamGroups or multiple compute streams?

I have lots of small kernels that run serially on TF’s compute stream as all the threads are using the same compute stream. I’d like to have each thread (from the intra pool) to use a dedicated compute stream so that small kernels can be executed simultaniously on the gpu.

Thanks

Eyal

---

<div class="post-metadata">

**Author:** ![Divya\_Sree\_Kayyuri](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/divya_sree_kayyuri/32/29791_2.png) [@Divya\_Sree\_Kayyuri](https://discuss.ai.google.dev/u/Divya_Sree_Kayyuri)\
**Post date:** [December 2, 2025, 6:51am UTC](https://discuss.ai.google.dev/t/multi-compute-streams-for-gpu/110288/3 "2025-12-02T06:51:06Z")

</div>

Hi @Eyal_Hirsch, Welcome to the Google AI Forum!  
TensorFlow, by default, uses a single compute stream per GPU device for kernel execution. Assigning different CUDA streams to different threads is not supported. Its not possible with python high level APIs. If you want to use multiple Cuda Streams explicitly, you can do it with low level C++ APIs. Thanks!

---

<div class="post-metadata">

**Author:** ![Eyal\_Hirsch](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/eyal_hirsch/32/9360_2.png) [@Eyal\_Hirsch](https://discuss.ai.google.dev/u/Eyal_Hirsch)\
**Post date:** [December 3, 2025, 6:01am UTC](https://discuss.ai.google.dev/t/multi-compute-streams-for-gpu/110288/5 "2025-12-03T06:01:34Z")

</div>

Hi @Divya_Sree_Kayyuri , thanks for the answer.

I see there’s a new experimental feature in TF for virtual devices.

I was able to get 3 cuda streams per virtual device (compute, h2d, d2h) and verified it in NVIDIA’s nsys profiler. Struggling to see if it actually can bring performance gains. Still no luck even though the profiler clearly shows overlapping compute kernels from different compute streams.

thanks

Eyal