# Distributed Training

**URL:** <https://discuss.ai.google.dev/t/distributed-training/22430>\
**Category:** TensorFlow\
**Tags:** gpu, training\
**Created:** [July 17, 2023, 7:49am UTC](https://discuss.ai.google.dev/t/distributed-training/22430 "2023-07-17T07:49:34Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kavya\_Saxena](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/kavya_saxena/32/16542_2.png) [@Kavya\_Saxena](https://discuss.ai.google.dev/u/Kavya_Saxena)\
**Post date:** [July 17, 2023, 7:49am UTC](https://discuss.ai.google.dev/t/distributed-training/22430/1 "2023-07-17T07:49:34Z")

</div>

I started doing distributed training (using 2 GPUs) in TensorFlow and followed the distributed training tutorial online. The code is getting executed, and it calculates the loss for a batch at each GPU, but the model updation is taking a lot of time. What could possibly be wrong?

---

<div class="post-metadata">

**Author:** ![Laxma\_Reddy\_Patlolla](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/laxma_reddy_patlolla/32/19312_2.png) [@Laxma\_Reddy\_Patlolla](https://discuss.ai.google.dev/u/Laxma_Reddy_Patlolla)\
**Post date:** [July 17, 2023, 6:28pm UTC](https://discuss.ai.google.dev/t/distributed-training/22430/3 "2023-07-17T18:28:35Z")

</div>

Hi @Kavya_Saxena ,

Could you please elaborate on the issue more and, if possible, the steps you are taking(please provide the code or reference link) ?

There could be various reasons for model updating, such as communication overhead, inefficient GPU utilization, hardware limitations, Batch Size, frequency of synchronization,etc.

Thanks.