Summary
We generate course narration with gemini-3.8-flash-tts and a replicated voice. The reference recording of that voice is in German. With Spanish or Russian input text, some generations contain German words or whole German passages, although the input text contains no German. Identical requests sometimes drift and sometimes do not.
We would like to know whether this is known behaviour and how to bind the output language for a replicated voice.
Setup
- Model:
gemini-3.8-flash-tts - Route: Batch API (
batches.createwith inline requests), Python SDKgoogle-genai2.x - Voice: replicated voice created with
voices.create(store=True)from a German reference recording and a verified consent recording by the same speaker, who is the owner of the voice - Output: default WAV,
finish_reasonSTOP in all test responses
Request structure (voice ID removed, speaker name replaced):
{
"contents": [{"role": "user", "parts": [{
"text": "<verbatim transcript in the target language>",
"speech_metadata": {"style": "Energetic, brisk Spanish YouTube explainer narration in <speaker>'s conversational speaking style. Fast, confident pace with momentum between sentences; short pauses, a brief pause only after key numbers. Lively sentence melody, never rushed or mumbled. Keep technical terms crisp."}
}]}],
"config": {
"response_modalities": ["AUDIO"],
"speech_config": {"voice_config": {"voice": "<replicated voice id>"}}
}
}
How we measure
- Each take is cut into 8-second windows.
whisper.cpp(large-v3) detects the language of each window without a language hint. - A take counts as drifted if at least one window is detected as German with p of at least 0.5.
- A CTC speech recogniser for the target language serves as a second check on selected takes.
- A speech recogniser forced to the target language hides this problem. One take with German detected in 6 of 6 windows still reached a word-sequence score of 0.94 against the Spanish text.
- Confirmation by ear: a native German speaker listened to samples, including the example below, and clearly heard German words in the Spanish audio.
Observed rates
First generation of each slide-length take, same voice and same style:
| Target language | Takes | Takes with drift | Rate |
|---|---|---|---|
| Spanish | 214 | 17 | 7.9 % |
| Russian | 202 | 6 | 3.0 % |
| English, French, Italian, Portuguese, Czech, Polish (smaller sample) | 252 | 0 | 0 % |
Further observations:
- In 20 of the 23 drifted takes, German starts in the first 8-second window.
- In 17 of the 23, the audio returns to the target language later in the same take.
- Longer takes are affected more often. Spanish: 10 of 68 from 600 characters, 3 of 117 from 300 to 599 characters, 4 of 29 below 300 characters.
- Drifted and clean takes occur in the same batch job.
- We also saw drift in Russian with an earlier, different style text (10 of 79 takes). Czech and Polish stayed low with that style (0 of 95, 1 of 83).
Input text is clean
We read the submitted text of the affected takes. It contains no German. Example (Spanish, excerpt):
La ventaja uno es la disciplina. El mecanismo que hay detrás es de lo más simple. Valoras la próxima señal con la sensación del último trade.
In the audio of the affected take, the opening passages are detected as German with p above 0.99, and German words are audible in them.
Test with repeated requests
Three known problem texts (two Spanish, one Russian), repeated on purpose. These texts were selected because they had drifted, so the rates are not production rates. All requests were accepted and returned audio.
| Variant | Takes | Takes with drift |
|---|---|---|
| Style as above, no language parameter | 9 | 3 |
Same style plus speech_config.language_code (es-ES, ru-RU) |
9 | 6 |
| Short style without language name | 9 | 9 |
| No style | 9 | 9 |
On two of the three texts only:
| Variant | Takes | Takes with drift |
|---|---|---|
| Style as above, no language parameter | 6 | 2 |
| Style translated into the target language | 4 | 4 |
language_code, no style |
4 | 3 |
Prebuilt voice Orus, style as above |
4 | 0 |
- The identical request drifted in one of three runs on each of the three texts, so the effect looks random per generation.
language_codewas accepted without error, but we saw no reduction (6 of 9 against 3 of 9, not statistically significant).- Removing the style made it clearly worse in this sample (9 of 9 against 3 of 9).
- The prebuilt voice result is only an indication. It is 4 takes, and 0 of 4 against 2 of 6 with the same style is not significant.
- Generating eight sentences of one text separately gave no drift (0 of 8). That is one text, and short clips are harder to classify.
Separate observation: in one of the 76 test requests, with the prebuilt voice, the complete text was spoken twice (111 s instead of about 56 s). We could not reproduce it in the other requests.
Second test: storage route, model and generation unit
Four new problem texts (two Spanish, two Russian), each repeated, plus two control texts. Same style text. All requests were accepted, 108 of 108 returned audio.
| Variant | Problem texts: takes with drift | Controls: takes with drift |
|---|---|---|
gemini-3.8-flash-tts, stored replicated voice, whole slide |
3 of 12 | 0 of 2 |
Same, stateless voice key (store=False) instead of stored voice |
3 of 12 | 0 of 2 |
gemini-3.8-flash-lite-tts, same stored voice |
1 of 12 | 0 of 2 |
gemini-3.8-flash-tts, stored voice, generated per step instead of per slide |
0 of 8 | 1 of 2 |
- The storage route makes no difference. Generating in shorter steps did not prevent the problem: per-clip measurement finds German in 1 to 3 of 10 takes. Flash-Lite drifts as well: its one drifted take is German in 6 of 6 windows.
- None of these differences is statistically significant at this sample size. The problem texts were selected because they had drifted, so these are not production rates.
Questions
- Is this known behaviour for replicated voices whose reference recording is in another language?
- What is the recommended way to bind the output language for a replicated voice?
- Does
language_codeapply togemini-3.8-flash-ttsand to replicated voices? The speech generation guide says the language is detected automatically, whileSpeechConfigin the API reference listslanguageCodewith a fixed list of values. That list does not include Czech (cs-CZ), although the model supports Czech. - Does the Interactions API field
speech_config[].languagebehave differently fromlanguage_code, and is it available on the Batch route? - Would a reference recording in the target language reduce this, and is one voice per language the intended use?
- Are repeated full passages a known issue, and is there a recommended check for them?
Audio samples
We do not attach audio publicly because it contains a cloned voice. We can share samples, request IDs and batch details privately with the Google team on request.