Dual GPU, one GPU fills up the other doesn't

TensorFlow General Discussion
,
1

After running into OOM errors using Tensorflow Object Detection API with an NVIDIA 3080 (10GB) I bought a 4090 (24GB). I am currently running both together, but I noticed that in high batch size runs, I’m using almost all the 3080 but varying amounts of the 4090. Ideally I’d like to use all of both cards to push the batch size as high as possible. I can’t seem to find a way to change the strategy so that the GPUs can take different loads. The Mirrored strategies seem to give each GPU the same amount of data to process during each split. Is there a way that one GPU can have more and the other less?

My machine and environment specs are as follows:

OS = Ubuntu 22.04
GPUs [0: 4090, 1: 3080]
python = 3.10.9
cudatoolkit = 11.2.2 (installed through anaconda)
cudnn = 8.1.0.77 (installed through anaconda)

GPU Memory usage during training|
GPU Memory usage during training|1913×890 157 KB

I’m fairly new to this, so any help is appreciated. If I’ve left out any useful information, please let me know and I’ll edit the post accordingly. Thanks in advance.

Derek

3

Hi @Derek_Boase, As per my knowledge while using a distributed strategy, each GPU receives the same amount of data based on the batch size provided, it might not be possible to send the different size of data for each GPU. Thank You.

5

the calculation must be parallelized, but with XLA