Live API / Realtime / Audio

Gemini API
1

Gemini Live API: Input transcription drops beginning of long pause-heavy user turns

Hi team,

We’re evaluating the Live API (gemini-3.1-flash-live-preview) for a real-time voice application and have observed what appears to be truncation of input transcription during long user turns containing natural pauses.

Environment

  • Model: gemini-3.1-flash-live-preview

  • Server-side activity detection enabled

  • Input and output audio transcription enabled

  • Default activity detection settings, except:

    • prefix_padding_ms = 300

Observed behaviour

When a user speaks for an extended period with occasional natural pauses (typically 1–3 seconds), the input transcription received by the application sometimes contains only the latter portion of the user’s speech. The beginning of the utterance appears to be missing, with the transcription starting partway through the response.

In our testing:

  • Total user speech duration was approximately 90–100 seconds.

  • Our client-side audio telemetry indicates approximately 90–100 seconds of speech was transmitted successfully.

  • Speech included several natural pauses between sentences.

  • The final input transcription contained only the latter part of the response rather than the full utterance.

  • Session logs showed an interrupted/resumed speech event during the turn.

  • The transcript begins mid-sentence, suggesting earlier content may not have been retained.

Questions

  1. Is this expected behaviour when server-side activity detection determines that a turn has ended and subsequently starts a new activity?

  2. Are there any limits on input transcription duration, buffer size, or retained audio that could result in earlier portions of speech being omitted?

  3. For users who may speak for extended periods with occasional 1–3 second pauses, what activity detection settings do you recommend (e.g. silence duration, start/end sensitivity)?

  4. Is there a supported approach for ensuring transcription continuity across pauses within a single logical user turn?

  5. For long-form spoken responses, would you recommend manual activity detection rather than automatic turn detection?

We’d appreciate any recommended configuration, implementation pattern, or best practice for handling long-form user speech while preserving complete transcription fidelity.

Happy to provide session IDs, timestamps, logs, and a minimal reproduction example if that would help with investigation.

Thanks in advance for any guidance.

2

Hello @AshishBhuta ,

  1. Yes, this is expected. When server-side Voice Activity Detection (VAD) detects silence for more than a few hundred ms, it commits the turn and starts generating a response. Subsequent speech is treated as a new turn.
  2. There are no turn-level buffer size limits causing this; it is entirely due to the VAD turn-splitting mentioned above.
  3. You can use the AutomaticActivityDetection config . For example, try setting ‘silence_duration_ms’ to 2000 (maximum allowed limit) and testing different values for startOfSpeechSensitivity, endOfSpeechSensitivity, etc. But please note that certain settings like increasing silence duration will increase response latency when the user finishes speaking.
  4. To ensure continuity: Adjust VAD settings (point 3) or Disable VAD and signal turn boundaries manually (point 5)
  5. Yes, manual control is better suited if pauses exceed 2 seconds. Configuration: 5.1 Disable Auto Voice Activity Detection: { “setup”: { “model”: “models/gemini-3.1-flash-live-preview”, “realtimeInputConfig”: { “automaticActivityDetection”: { “disabled”: true } } } } 5.2 Signal Turn Boundaries on the stream: 5.2.1 Start speaking: Send {“realtimeInput”: {“activityStart”: {}}} 5.2.2 Finish speaking: Send {“realtimeInput”: {“activityEnd”: {}}} (this triggers model generation).