[Bug] gemini-3.7-flash: 8 of 17 30-min audio transcription calls return HTTP 200 with zero output tokens, full input billed — stable 47% across three sessions (Aug 14/17/23), controls clean

Gemini API
, , ,
1

TL;DR: On 30-minute audio transcription, gemini-3.7-flash returns an HTTP 200 with zero output tokens on 47% of calls — 8 of 17 billed calls across three sessions on Aug 14, 17, and 23 — while the full audio input (~45K tokens) is billed each time ($0.034/failure at introductory sync pricing). The empty responses carry three different finish reasons, including a plain STOP, so they are indistinguishable from success without an explicit zero-length check. gemini-3-flash-preview, called interleaved in the same minutes on the same file, is 9/9 clean.

Setup

  • Models: gemini-3.7-flash (challenger), gemini-3-flash-preview (control), Python SDK google-genai 1.74.0, sync generate_content, default generation config, audio via Files API.

  • Audio: one 30-minute MP3 (1800.144 s, ~45,001 audio tokens), a prefix cut of a publicly available lecture recording: https://shiurim.yutorah.net/2009/1053/732141.MP3 cut with ffmpeg -i src.mp3 -t 1800 -c copy 30min.mp3. Same cut for every call, every model, every session.

  • Prompt (identical every call):

    Transcribe this entire Torah shiur audio verbatim from start to finish. Rules:

    • Output plain text only. No timestamps, no speaker labels, no commentary.
    • Preserve Hebrew, Aramaic, and Yiddish terms in their spoken (romanized) form.
    • Break into readable paragraphs at natural pauses.
    • Do not summarize. Do not skip any portion of the audio. Cover the FULL recording end to end.

The empty-response data

Every row below is a billed HTTP 200. “EMPTY” = candidates yields no text and usage_metadata reports 0 output tokens after ~45K input tokens billed.

session (UTC) billed calls EMPTY finish reasons on the EMPTYs
2026-08-14, 18:36–19:14 5 2 MALFORMED_RESPONSE¹, NO_CANDIDATES
2026-08-17, 20:39–20:52 6 3 NO_CANDIDATES ×2, MALFORMED_RESPONSE
2026-08-23, 17:01–17:13 6 3 STOP (with 0 output tokens), NO_CANDIDATES ×2
total 17 8 (47%)

¹ also raised UserWarning: MALFORMED_RESPONSE is not a valid FinishReason from the SDK — the value is outside the SDK’s own enum.

This is not a capacity artifact. Sample from Aug 23 — the same file, same prompt, empty and clean responses interleaved seconds apart, and the control model clean in the same window:

17:01:19  3.7-flash          OK     30,017 chars

17:01:50 3.7-flash EMPTY 0 chars, finishReason STOP

17:02:04 3-flash-preview OK 28,943 chars <- control

17:02:40 3.7-flash EMPTY 0 chars, NO_CANDIDATES

17:03:14 3.7-flash OK 30,981 chars

17:03:40 3-flash-preview OK 30,010 chars <- control

17:03:49 3.7-flash EMPTY 0 chars, NO_CANDIDATES

17:04:33 3.7-flash OK 31,569 chars

There were zero 503s in the Aug 17 and Aug 23 sessions (the Aug 14 session was launch week and is reported net of capacity sheds). The rate has not moved across nine days: 2/5, 3/6, 3/6.

Why this is worse than a failure

  1. It bills. Each empty response charges the full ~45K-token audio input. At a 47% rate, the effective input cost of this model on this workload is ~1.9× list price before any output is priced in — which more than erases the introductory discount.
  2. It masquerades as success. An HTTP 200 with finishReason: STOP and zero output is only detectable by explicitly checking output length / output token count. A 5xx would be retried by any sane client and not billed; this is neither.

This appears related to the “rare 200 responses with empty candidates and finishReason: STOP” mentioned in Gemini-3.7-flash: schema-constrained JSON output degenerates into repeated "0" until maxOutputTokens (regression vs gemini-3-flash-preview) — on this audio workload it is not rare.

A secondary observation from the same controlled runs

Non-Latin script leakage, on both discounted models, despite an explicit romanization instruction. The prompt requires romanized output (see above). Starting Aug 17 — never before that date on identical input — completions began interleaving Hebrew script: gemini-3.7-flash in one completion per session (3,890 and 3,184 chars, ~10% of the transcript, scattered across ~400 short spans, e.g. the תנאים had a tradition, how to read the פסוק); gemini-3.6-flash at lower volume (458/7/7 chars on Aug 17, 12 on Aug 23). That two different models began doing this on the same date, while gemini-3-flash-preview stayed at zero Hebrew-script characters in 9/9 runs under the same instruction, suggests a shared serving-side change rather than a per-model quirk. The transcribed content looks accurate — it is an instruction-compliance regression, not hallucination.

(We also have data on structural run-to-run variance for gemini-3.6-flash on this audio; leaving it out of scope here — can open a separate topic if useful.)

Ask

  1. Confirm whether zero-output-token 200s are a known defect on gemini-3.7-flash, and whether input billing on them is intended.
  2. If it is a defect: is there a fix ETA, and is there a path to billing credit for the affected calls?
  3. Guidance on the script-leakage regression: is romanization instruction-following a known change in the 3.6/3.7 serving path since ~Aug 17?

Happy to share per-call usage_metadata, request timestamps for billing correlation, and the full run logs — everything above is reproducible from the public audio URL and the prompt as posted.

2

Still seeing this on 3.7 Flash as of Sept 7. Can someone from the Gemini API team confirm whether this is a known issue?