Gemini 3.8 Flash TTS with a replicated voice sometimes speaks German words in Spanish and Russian text

Summary

We generate course narration with gemini-3.8-flash-tts and a replicated voice. The reference recording of that voice is in German. With Spanish or Russian input text, some generations contain German words or whole German passages, although the input text contains no German. Identical requests sometimes drift and sometimes do not.

We would like to know whether this is known behaviour and how to bind the output language for a replicated voice.

Setup

  • Model: gemini-3.8-flash-tts
  • Route: Batch API (batches.create with inline requests), Python SDK google-genai 2.x
  • Voice: replicated voice created with voices.create(store=True) from a German reference recording and a verified consent recording by the same speaker, who is the owner of the voice
  • Output: default WAV, finish_reason STOP in all test responses

Request structure (voice ID removed, speaker name replaced):

{
  "contents": [{"role": "user", "parts": [{
    "text": "<verbatim transcript in the target language>",
    "speech_metadata": {"style": "Energetic, brisk Spanish YouTube explainer narration in <speaker>'s conversational speaking style. Fast, confident pace with momentum between sentences; short pauses, a brief pause only after key numbers. Lively sentence melody, never rushed or mumbled. Keep technical terms crisp."}
  }]}],
  "config": {
    "response_modalities": ["AUDIO"],
    "speech_config": {"voice_config": {"voice": "<replicated voice id>"}}
  }
}

How we measure

  • Each take is cut into 8-second windows. whisper.cpp (large-v3) detects the language of each window without a language hint.
  • A take counts as drifted if at least one window is detected as German with p of at least 0.5.
  • A CTC speech recogniser for the target language serves as a second check on selected takes.
  • A speech recogniser forced to the target language hides this problem. One take with German detected in 6 of 6 windows still reached a word-sequence score of 0.94 against the Spanish text.
  • Confirmation by ear: a native German speaker listened to samples, including the example below, and clearly heard German words in the Spanish audio.

Observed rates

First generation of each slide-length take, same voice and same style:

Target language Takes Takes with drift Rate
Spanish 214 17 7.9 %
Russian 202 6 3.0 %
English, French, Italian, Portuguese, Czech, Polish (smaller sample) 252 0 0 %

Further observations:

  • In 20 of the 23 drifted takes, German starts in the first 8-second window.
  • In 17 of the 23, the audio returns to the target language later in the same take.
  • Longer takes are affected more often. Spanish: 10 of 68 from 600 characters, 3 of 117 from 300 to 599 characters, 4 of 29 below 300 characters.
  • Drifted and clean takes occur in the same batch job.
  • We also saw drift in Russian with an earlier, different style text (10 of 79 takes). Czech and Polish stayed low with that style (0 of 95, 1 of 83).

Input text is clean

We read the submitted text of the affected takes. It contains no German. Example (Spanish, excerpt):

La ventaja uno es la disciplina. El mecanismo que hay detrás es de lo más simple. Valoras la próxima señal con la sensación del último trade.

In the audio of the affected take, the opening passages are detected as German with p above 0.99, and German words are audible in them.

Test with repeated requests

Three known problem texts (two Spanish, one Russian), repeated on purpose. These texts were selected because they had drifted, so the rates are not production rates. All requests were accepted and returned audio.

Variant Takes Takes with drift
Style as above, no language parameter 9 3
Same style plus speech_config.language_code (es-ES, ru-RU) 9 6
Short style without language name 9 9
No style 9 9

On two of the three texts only:

Variant Takes Takes with drift
Style as above, no language parameter 6 2
Style translated into the target language 4 4
language_code, no style 4 3
Prebuilt voice Orus, style as above 4 0
  • The identical request drifted in one of three runs on each of the three texts, so the effect looks random per generation.
  • language_code was accepted without error, but we saw no reduction (6 of 9 against 3 of 9, not statistically significant).
  • Removing the style made it clearly worse in this sample (9 of 9 against 3 of 9).
  • The prebuilt voice result is only an indication. It is 4 takes, and 0 of 4 against 2 of 6 with the same style is not significant.
  • Generating eight sentences of one text separately gave no drift (0 of 8). That is one text, and short clips are harder to classify.

Separate observation: in one of the 76 test requests, with the prebuilt voice, the complete text was spoken twice (111 s instead of about 56 s). We could not reproduce it in the other requests.

Second test: storage route, model and generation unit

Four new problem texts (two Spanish, two Russian), each repeated, plus two control texts. Same style text. All requests were accepted, 108 of 108 returned audio.

Variant Problem texts: takes with drift Controls: takes with drift
gemini-3.8-flash-tts, stored replicated voice, whole slide 3 of 12 0 of 2
Same, stateless voice key (store=False) instead of stored voice 3 of 12 0 of 2
gemini-3.8-flash-lite-tts, same stored voice 1 of 12 0 of 2
gemini-3.8-flash-tts, stored voice, generated per step instead of per slide 0 of 8 1 of 2
  • The storage route makes no difference. Generating in shorter steps did not prevent the problem: per-clip measurement finds German in 1 to 3 of 10 takes. Flash-Lite drifts as well: its one drifted take is German in 6 of 6 windows.
  • None of these differences is statistically significant at this sample size. The problem texts were selected because they had drifted, so these are not production rates.

Questions

  1. Is this known behaviour for replicated voices whose reference recording is in another language?
  2. What is the recommended way to bind the output language for a replicated voice?
  3. Does language_code apply to gemini-3.8-flash-tts and to replicated voices? The speech generation guide says the language is detected automatically, while SpeechConfig in the API reference lists languageCode with a fixed list of values. That list does not include Czech (cs-CZ), although the model supports Czech.
  4. Does the Interactions API field speech_config[].language behave differently from language_code, and is it available on the Batch route?
  5. Would a reference recording in the target language reduce this, and is one voice per language the intended use?
  6. Are repeated full passages a known issue, and is there a recommended check for them?

Audio samples

We do not attach audio publicly because it contains a cloned voice. We can share samples, request IDs and batch details privately with the Google team on request.

Correction and update: German drift also occurs in Portuguese, Italian and English

Correction

Our report said 0 of 252 takes drifted across English, French, Italian, Portuguese, Czech and Polish. That number was correct for the sample, but it covered only the first lessons of the course and is not representative. The audio itself did not change: all 378 takes of that first sample have the same checksums in the later full run.

Same measure as before (8-second windows, language detected without a hint, German with p of at least 0.5), now on the full set of first generations:

Target language Takes Takes with drift Rate
Portuguese 236 22 9.3 %
Spanish 236 17 7.2 %
Russian 236 6 2.5 %
Italian 236 4 1.7 %
English 236 2 0.8 %
French 236 0 0 %
Czech 226 0 0 %
Polish 226 0 0 %
  • These replace our earlier Spanish (17 of 214) and Russian (6 of 202) figures.
  • We re-measured all flagged Portuguese, Italian and English takes separately and got the same counts.
  • A stricter per-clip check flags more takes (Portuguese 25, Italian 10, English 4). We report the window measure to stay comparable.
  • In 43 of the 51 drifted takes, German starts in the first 8-second window.

New tests

All tests used gemini-3.8-flash-tts with the stored replicated voice. The texts were selected because they had drifted, so these are not production rates. Results come from the window measure; only some samples were checked by ear.

temperature and seed. Against 3 of 12 takes with default settings: temperature 0.3 gave 4 of 12, temperature 0 gave 1 of 4, and temperature 0.3 with a fixed seed gave 5 of 12. A fixed seed made only some repeats nearly identical and did not prevent drift.

Calmer pace in the style text. 5 of 12, no reduction.

Short opening sentence stating the language, spoken before the text and excluded from measurement. A first test gave 0 of 12 against 3 of 12. A confirmation with a fixed pass criterion failed: 9 of 40 against 14 of 40 (one-sided Fisher p = 0.16). On one Russian text it drifted in 5 of 5 takes.

Short neutral opening sentence in the target language, without naming the language. In the test above this was descriptive only: 0 of 16 against 9 of 20 on the same four texts. We then ran a separate test with a pass criterion fixed in advance, on eight other texts (Portuguese, Spanish, Russian, Italian), five runs each:

Variant Takes Takes with drift
No opening sentence, measured from 1.8 s 40 20
Neutral opening sentence, measured from its end 40 3

The difference is large (one-sided Fisher p below 0.0001), but the test did not pass: the criterion also required at most 2 of 40 drifted takes, and we missed it by one take. All three drifted takes are on one Portuguese text, and two of them show German in a single window only. We therefore do not call this a confirmed workaround, and nobody has listened to these takes yet.

Spoken text beyond the end of the input

These observations are based on transcripts from a speech recogniser and on audio duration. Nobody has listened to these takes.

  • One Spanish take (66 s against about 40 s for the other runs of the same text) contains the full input and then continues with about 25 seconds of content that is not in the input, including figures we did not supply. It is not a repeat of the input, and we did not find the wording in the slide texts of that lesson.
  • One Portuguese take with the neutral opening sentence runs 273 s against a median of 41 s for that text. German is detected in all 34 windows, and the transcript is about four times the length of the input.
  • In the stock of 2,124 first generations in nine languages, seven takes run more than 1.3 times their expected duration. In six of them (Russian, Portuguese, Italian twice, Polish twice; 1.78 to 1.98 times) the transcript has about twice the words of the input, and most of the input appears in each half, so the input appears to be spoken twice. The seventh take (1.33 times) has no extra words in its transcript.

What these numbers do not depend on

Our pipeline later cuts each take into clips at step boundaries, and that cutting has produced errors of its own. None of the window rates, drift counts or durations above depend on it: they are measured on the complete audio returned by the API, before any cutting. The one exception is the shift in the opening-sentence tests, where we skip the opening sentence or the first 1.8 s for measurement only. The stricter per-clip counts in the correction above do run on cut clips, which is why we do not use them for the rates.

Questions

In addition to our earlier questions:

  1. Is a language other than the reference language expected to be less stable for replicated voices, and why would rates differ so much between languages (Portuguese and Spanish against French, Czech and Polish)?
  2. Can the model add content to a verbatim transcript or speak it twice, and is there a way to prevent or detect this?
  3. Could a short sentence at the start of the input affect the language behaviour of the rest of the take, and if so, why?

Audio samples, request IDs and batch details remain available privately to the Google team on request. We do not attach them publicly because they contain a cloned voice.

Send code , prompt and output

The request structure, the style prompt and the measurements are in my first post and the update above. Audio samples contain a replicated voice, so I can only share them privately with the Google team. Request IDs and batch details are available to them on request.

Update with two further tests. All on gemini-3.8-flash-tts with the stored replicated voice, on texts selected because they had drifted, so these are not production rates. German is counted in 8-second windows at p of at least 0.5.

Correction to my update: The API reference states that temperature is ignored from Gemini 3.6 Flash onward, so our temperature comparison likely carries no conclusion.

Neutral opening sentence, third test. With a limit set in advance after the earlier miss (p below 0.01, at most 4 of 40): 2 of 40 against 12 of 40, p = 0.0032, passed. The limit was loosened after that miss, so this is no independent confirmation at the original limit. Both Russian texts showed no drift in either variant.

Style text and language code. Our style text contained the speaker’s first name, which the speech guide advises against. Criterion fixed in advance: p below 0.0167 and at most 4 of 40, against the current request.

Variant Takes with drift of 40 p Criterion
Current request (name in style) 18
Style without the name 8 0.015 not passed
Style without the name, plus language_code 4 0.0004 passed
Style without the name, plus neutral opening sentence 4 0.0004 passed

The effect of language_code alone is not established: 4 against 8 of 40 is not significant (p = 0.35). Sixteen direct Interactions calls with speech_config[].language gave 1 drifted take, descriptive only. In a blind sample of 8 takes, all 4 with the name removed and language_code were judged clean and preferred; of the 4 current takes, 2 had German colouring and 2 audible German words.

Portuguese remains the weakest language: one text still drifted in 3 of 5 takes. We still see this as a mitigation, not a fix, and our questions remain open. Why would a name in the style or a language code affect the spoken language of a verbatim transcript?