Gemini Flex API always throws errors

Gemini API
,
1

I used flex inference successfully with gemini-3.1-pro-preview but this week not a single request came through. I used the same script as last week and all retries ended in errors.

Mostly:

503 UNAVAILABLE. {‘error’: {‘code’: 503, ‘message’: ‘This model is currently experiencing high demand. Spikes in demand are usually temporary. Please try again later.’, ‘status’: ‘UNAVAILABLE’ } }

I have 10 retries with exponential backoff, but it never resolves. No matter which time of day I try it. When I switched to service tier standard it immediately went through. So, currently I am using this as a fallbak c.

I was completely fine with the increased latency, as long as the request go through at some point. But throwing request after request against the flex api and never getting anything but an error back feels pretty useless.

Is this to be expected? Did anything change about the flex api? Does it work fine for oth er peop le?

I understand this feature is still in preview, so stuff changes. But I just want to make sure, it is not something that I did wr ong.

2

I am trying to make a similar solution , but currently using the standard api , I was thinking to switch to flex inference , are the errors still occuring ? And do you have an idea that can I use it for large volume workload ?

3

For for large volume workload the batch API is probably a better pick? Assuming you don’t care about latency. I believe the batch API is more reliable.

The errors are very variable. It has gotten better, but you definitely need the standard tier as a fallback. I think, you do not get charged for token costs when the request fails, so I am still using it to at least sometimes save some money on my video analysis (not time-sensitive).
Currently trying the flex tier once, and it it fails I catch the error and repeat the request with the standard tier.

4

Batch would take 24 hours , that is not ideal for my usecase . I was thinking the same to use standard as fallback with context caching .
Do you have an idea like out of 100 requests or 1000 requests , how much percentage or number requests get failed for flex inference on an average ?