I’m using Gemini 2.5 Flash as the LLM in my voice agent pipeline. Its responses are 300-400ms and well suited for the use case. My production environment is Australia.
I was really disappointed to see that 3.5 Flash has considerably higher inference times, 600-700ms. Additionally, Australia has not been given a deployment, so the latency per request is closer to 700-800ms.
Considering the total time for a conversation turn for a voice agent should be ~800ms, this makes 3.5 Flash unsuitable: 5x the cost for 2x the latency.
It’s really a shame that 2.5 Flash can’t be maintained because it fills an entirely different use case from other Gemini models. It is without a doubt the strongest low-latency model in APEC by a large margin.
I’m wondering if anyone else is running into this problem and what providers you are considering switching to after 2.5 is sunset in October? I feel like I’ve looked at every model available on Bedrock and Foundry at this point, but nothing seems to come close to 2.5 Flash.