gemini-3.7-flash: 2-27s latency for a one-word reply on the Gemini API — persists on priority tier; thinkingBudget: 0 intermittently ignored (empty candidates)
WHEN: 2026-09-01, ~17:15-17:35 UTC
FROM: our production backend runs in Convex’s cloud environment; the results below were measured via direct curl (no proxy), and the same signature reproduces through the Vercel AI SDK (ai@7.0.83, @ai-sdk/google@4.0.56, @ai-sdk/google-vertex@5.0.68) and in our backend logs.
BILLING: prepaid key, priority-eligible (confirmed by the served-tier echo below).
REQUEST (identical every call): generateContent, prompt “Reply with the single word: ok”, generationConfig: { maxOutputTokens: 50, thinkingConfig: { thinkingBudget: 0 } }
TIER SELECTION: Gemini API → top-level serviceTier body field. Vertex → X-Vertex-AI-LLM-Shared-Request-Type header.
RESULTS (total seconds per call, 3 calls per row)
Gemini API (generativelanguage.googleapis.com, v1beta):
- gemini-3.7-flash, no tier: 6.07s, 13.18s, 26.58s — served standard; thoughts 0/47/0
- gemini-3.7-flash, standard: 17.01s, 6.91s, 4.89s — served standard; thoughts 0/47/0
- gemini-3.7-flash, PRIORITY: 8.61s, 2.12s, 11.52s — served PRIORITY (header + usageMetadata both confirm); thoughts 47/0/45
- gemini-3.5-flash, no tier: 0.96s, 1.03s, 0.95s — served standard; thoughts 0
Vertex AI (aiplatform.googleapis.com, v1beta1, location global):
- gemini-3.7-flash, no tier: 1.95s, 2.24s, 2.00s — thoughts 45/0/0
- gemini-3.7-flash, standard: 3.75s, 3.38s, 1.42s — thoughts 47/47/0
- gemini-3.7-flash, priority: 2.60s, 1.91s, 2.03s — thoughts 47/0/45 (one call returned usageMetadata.serviceTier “standard”; the other eight Vertex calls carried no serviceTier field at all)
- gemini-3.5-flash, no tier: 0.84s, 0.74s, 0.87s — thoughts 0
Vertex regional endpoints, same request (gemini-3.7-flash, no tier):
- global: 1.68-2.30s
- us multi-region (aiplatform.us.rep.googleapis.com): 1.50-4.17s
OBSERVATIONS
-
gemini-3.7-flash latency on the Gemini API is degraded and highly variable: 2.1-26.6s for a one-word reply. Controls isolate it to this model on this surface: gemini-3.5-flash on the same key answers in about 1s, and the same model on Vertex (global) answers in 1.4-3.8s.
-
Priority tier does not help: priority requests are confirmed served priority (x-gemini-service-tier: priority response header plus usageMetadata.serviceTier “priority”, HTTP 200) and still take 2.1-11.5s. That is not “consistent low-latency” behavior, at a 1.8x price.
-
thinkingConfig.thinkingBudget: 0 is intermittently ignored on gemini-3.7-flash (both surfaces, all tiers): thoughtsTokenCount comes back 45-47 on roughly half the calls. When it happens, the thinking consumes the maxOutputTokens: 50 budget and the response can carry an empty candidates text — a one-word prompt returns nothing.
-
Feature request: Vertex has no reliable served-tier echo — no tier response header, and usageMetadata normally carries only trafficType “ON_DEMAND”. Oddly, exactly one Vertex call (priority header) DID return usageMetadata.serviceTier “standard” while the other eight calls carried no field at all. With Priority PayGo billed at a premium, there is no response-level way to verify the queueing actually received.
We saw the same latency signature (high variance plus ignored thinkingBudget) on gemini-3.7-flash during the week of 2026-08-24; it recovered, and has now returned.