Title suggestion
gemini-3.1-flash-tts-preview generates near-silent audio despite finish_reason=STOP (not a safety block)
Post body
I’m using gemini-3.1-flash-tts-preview via the API (client.models.generate_content, non-streaming) to generate short voice-over lines (5-20 seconds of narrated French text) for a daily video project. Fixed settings used across all calls:
temperature = 0.2voice_name = "Achird"(prebuilt voice)response_modalities = ["audio"]max_output_tokenscalculated dynamically per line (generous margin, never the limiting factor in practice)safety_settingsset toBLOCK_ONLY_HIGHfor DANGEROUS_CONTENT, HARASSMENT, HATE_SPEECH
Every prompt follows a fixed structure: an “Audio Profile” section (voice character description), a “Director’s note” section (style/pace/accent), a “Scene” section (short scene description, changes per line), a “Sample Context” section (fixed project context), and a “Transcript” section (the text to read, changes per line).
The problem
On a real test batch of 18 short lines, the one-shot success rate is very low. Out of 14 lines processed in the last run: only 6 succeeded (some on the first try, others after 2-4 retries), and 8 failed even after 4 attempts.
The dominant failure mode: the API returns a “complete” and normal response (finish_reason = STOP, not MAX_TOKENS, not SAFETY), with an audio stream whose total duration roughly matches what was expected for the text — but analyzing the signal’s RMS energy reveals that most of that stream is near-total silence. Concrete example: for a line expected to take ~6 seconds of speech, the returned file is indeed ~17 seconds long, but only 0-2 seconds actually contain speech — the rest is silent padding. This isn’t detectable via finish_reason: the API considers the generation to have completed normally.
This happens on completely mundane content too (e.g. a line about a marriage proposal, a line about a boat escape) — so it doesn’t appear to be tied to a safety filter on the content (already relaxed via safety_settings, with no change in how often this specific issue occurs).
A second, rarer failure mode: finish_reason = SAFETY on lines containing typical video-game violence vocabulary (threats, weapons) — reduced with safety_settings but not fully eliminated.
What we’ve already tried client-side (no effect on how often this happens)
- Capping
max_output_tokensper line — didn’t reduce the frequency of silence (the cap is almost never actually hit in these cases anyway). - Detecting and auto-rejecting mostly-silent responses via RMS analysis, then retrying — works as a safety net, but often needs 3-4 attempts per line to get a usable result, which is neither cost-efficient nor practical for daily production given the 100 requests/day quota for this model.
- Relaxing
safety_settings— reduced explicitSAFETYblocks somewhat, no effect on the silence issue.
Questions
- Is this a known issue with
gemini-3.1-flash-tts-preview(silent padding after a few spoken words,finish_reason=STOPdespite near-empty content)? - Could
temperature = 0.2(set low on purpose to stabilize voice timbre across generations) actually be encouraging this degenerate behavior? Is there a better parameter combination (top_p,top_k, seed) to get voice consistency without triggering this silence issue? - Could the prompt structure (Audio Profile / Director’s note / Scene / Sample Context / Transcript sections, instructions in English but the text-to-read in French) be a factor? Is there a recommended prompt format for this model specifically for short narration?
- Is there a way to detect this issue directly from the API response (a field, a metric) instead of having to analyze the audio ourselves after the fact?
- Is the 100 requests/day per-model quota negotiable for this preview model, or is it a fixed ceiling independent of billing tier?
Happy to share more logs/examples if useful. Thanks!