- Target Services / Models:
gemini-3.8-flash-lite-ttsgemini-3.8-flash-tts- (Compared against baseline:
gemini-3.1-flash-tts-preview)
- SDK / API:
@google/genai(Node.js / REST API) - Date: October 2026
- Severity: Functional Defect / Documentation Contradiction
1. Executive Summary
Official Google Gemini 3.8 TTS documentation instructs developers to strictly keep part.text as a verbatim transcript and place all delivery and character directions into speech_metadata.style. The documentation explicitly warns:
“Do not include ‘Audio Profiles’ or ‘Director’s Notes’ in the text field, as these will be read aloud. Place these instructions strictly in
speech_metadata.style.”
Empirical testing reveals two critical findings:
speech_metadata.styleis completely ineffective for character acting, vocal texture, and persona styling: Even with explicit pirate directions or detailed physical vocal descriptors ("rough, gravelly, raspy old sea captain, gruff, enthusiastic shouting"), the synthesizer only applies minor prosodic adjustments (volume, pace). The base voice (Sadachbia,Puck, etc.) remains an unaltered, young, clean, corporate narrator voice without any gravel, rasp, or pirate cadence.- Placing a structured
# Audio Profileinpart.textactually works and does NOT get read aloud: Contradicting the documentation warning, the model reliably parses the boundary instruction (“Read the following transcript based on the audio profile”) and synthesizes audio only for the transcript. Crucially, this structured prompt triggers the generative vocal acting capabilities of the LLM, producing authentic grit, rasp, pirate inflection, and theatrical delivery.
2. Steps to Reproduce: The Non-Working Official Pattern
Request Payload (speech_metadata.style)
Using the officially documented parameter:
{
"model": "gemini-3.8-flash-lite-tts",
"contents": [
{
"parts": [
{
"text": "Let's sail and find that treasure! But... which way to go? ",
"speech_metadata": {
"style": "speak as a Pirate, express Excited emotion, use gruff vocal delivery"
}
}
]
}
],
"config": {
"temperature": 1,
"responseModalities": ["AUDIO"],
"speechConfig": {
"voiceConfig": {
"prebuiltVoiceConfig": { "voiceName": "Sadachbia" }
}
}
}
}
We also tested explicit, low-level descriptive styles:
"speech_metadata": {
"style": "rough, gravelly, raspy old sea captain, gruff, enthusiastic shouting"
}
Observed Behavior
- The generated audio sounds like a polite, 20-something modern podcast host / assistant reading words with a slight volume boost.
- Zero rasp, zero vocal fry, zero gravel, zero pirate dialect, zero age morphing.
- The
speech_metadata.styleparameter behaves merely as post-processing volume/pacing tags on top of an immutable, fixed prebuilt voice profile.
3. The Working Solution: Structured # Audio Profile in part.text
To achieve authentic character styling, the directions must be passed directly to the language model as structured context within part.text.
Working Request Payload
{
"model": "gemini-3.8-flash-lite-tts",
"contents": [
{
"parts": [
{
"text": "Read the following transcript based on the audio profile.\n\n# Audio Profile\n\n## Language:\n Speak in en-US.\n\n## Character:\n Speak as a Pirate.\n\n## Emotion:\n Express Excited emotion.\n\n## Delivery:\n Use gruff vocal delivery.\n\n## Transcript:\nLet's sail and find that treasure! But... which way to go? "
}
]
}
],
"config": {
"temperature": 1,
"responseModalities": ["AUDIO"],
"speechConfig": {
"voiceConfig": {
"prebuiltVoiceConfig": { "voiceName": "Sadachbia" }
}
}
}
}
Observed Behavior & Results
- No Prompt Leakage: The model never speaks the profile or prompt aloud. It starts vocalization precisely at “Let’s sail…”.
- Generative Vocal Acting Active:
- The model actively alters vocal timbre, lowering register and introducing vocal rasp and grit appropriate for a pirate.
- Natural theatrical pauses and pacing are observed around the ellipsis (“But… which way to go?”).
- Generates pirate inflection and swashbuckling cadence.
4. Model Comparison Findings: Gemini 3.1 Preview vs. Gemini 3.8 Flash-Lite
When using the structured # Audio Profile prompt across models:
| Feature / Dimension | Gemini 3.1 Flash Preview (gemini-3.1-flash-tts-preview) |
Gemini 3.8 Flash-Lite (gemini-3.8-flash-lite-tts) |
|---|---|---|
| Character Timbre Morphing | Outstanding. Dynamically adds deep rasp, age, gravel, and guttural textures (Pirate, Dragon, Monster, Grandma). | Moderate. Noticeable improvement over vanilla, but anchors closer to base speaker persona. |
| Theatrical Timing & Pauses | Superior. Natural pauses, hesitations, and dramatic phrasing. | Brisker. Reads ~30–40% faster with fewer dramatic pauses. |
Response to speech_metadata.style |
N/A (earlier preview pipeline). | Fails. Does not activate character acting. |
Response to # Audio Profile Prompt |
Full acting performance. | Engages acting module, but less intense timbre transformation than 3.1. |
5. Impact & Recommendations for Google Gemini Team
- Fix
speech_metadata.stylerouting: Ifspeech_metadata.styleis intended to replace prompt-based direction, its content must be fed into the multimodal generation stage rather than treated solely as downstream DSP prosody modifiers. - Revise Documentation: Remove the misleading warning claiming that
# Audio Profileor direction blocks inpart.textwill always be spoken aloud. When prefixed with"Read the following transcript based on the audio profile.", the model reliably separates metadata from transcript. - Preserve Vocal Expressiveness in 3.8: Retain the rich, generative voice-morphing capability that makes
gemini-3.1-flash-tts-previewdramatically superior for interactive storytelling, audiobooks, and character acting.