Summary
With automatic activity detection (server VAD) enabled, the Live API cuts the model’s turn and sends serverContent.interrupted when it detects user speech during generation — this is the barge-in signal a real-time voice app relies on to stop talking over the user.
However, interrupted is only ever emitted on a speech onset that begins after the model’s turn has started. If the user is already speaking at the moment the model’s turn begins, the server never sends interrupted, and the model’s audio plays to completion — the assistant talks over the user and cannot be cut off. The server clearly receives the audio (it returns inputTranscription for that same speech), but no interrupted is produced.
Environment
- Model:
gemini-3.1-flash-live-preview(Live API,client.aio.live) - SDK:
google-genai2.10.0 (Python) - Config:
response_modalities=["AUDIO"], automatic activity detection enabled (default) - Continuous 16 kHz mono PCM realtime audio via
send_realtime_input(open-mic / handsfree)
Expected behavior
In both cases below, the user’s speech should interrupt the model:
- A — model is generating, user starts speaking (fresh onset). →
interruptedfires.
- B — user is already speaking, then the model’s turn begins over them. → should also interrupt.
Actual behavior
- A interrupts reliably in ~0.2 s.
- B never emits
interrupted. The model generates its entire turn (20+ s of audio) over the continuously-speaking user. The user’s speech is transcribed by the server (inputTranscriptionreturns it), so the audio is being received and recognized — but no barge-in signal is produced.
There appears to be no other server signal for start/stop of user activity that a client could use instead, so once this state is entered the client has no way to know it should stop playback.
Minimal reproducible example (MRE)
pip install google-genai
export GOOGLE_API_KEY=YOUR_KEY
# any real 16 kHz mono PCM speech clip; on macOS:
say --data-format=LEI16@16000 -o speech.wav \
"Wait, are you sure about that? I really do not think that is correct."
python mre.py
# mre.py
import asyncio, wave
from google import genai
from google.genai import types
MODEL = "gemini-3.1-flash-live-preview"
RATE = 16000
CHUNK = RATE // 10 * 2 # 100 ms of int16
SILENCE = b"\x00" * CHUNK
LONG_TURN = "Count slowly upward from one, about one number per second, and keep going."
def load_chunks(path="speech.wav"):
w = wave.open(path, "rb")
assert (w.getframerate(), w.getnchannels(), w.getsampwidth()) == (RATE, 1, 2), \
"need 16 kHz mono int16"
pcm = w.readframes(w.getnframes())
return [pcm[i:i + CHUNK] for i in range(0, len(pcm), CHUNK)]
CONFIG = types.LiveConnectConfig(
response_modalities=["AUDIO"],
input_audio_transcription=types.AudioTranscriptionConfig(), # to show the server hears us
realtime_input_config=types.RealtimeInputConfig( # auto VAD (this is the default)
automatic_activity_detection=types.AutomaticActivityDetection(disabled=False)),
)
async def run(label, user_already_speaking):
chunks = load_chunks()
st = {"speak": False, "i": 0, "stop": False, "audio_s": 0.0,
"interrupted": False, "transcribed": False, "gen": False}
async with genai.Client().aio.live.connect(model=MODEL, config=CONFIG) as s:
async def mic(): # continuous realtime audio, like an open mic
while not st["stop"]:
buf = chunks[st["i"] % len(chunks)] if st["speak"] else SILENCE
if st["speak"]:
st["i"] += 1
await s.send_realtime_input(audio=types.Blob(
data=buf, mime_type=f"audio/pcm;rate={RATE}"))
await asyncio.sleep(0.1)
async def recv():
while not st["stop"]: # NOTE: receive() returns per-turn — must re-enter it
async for m in s.receive():
sc = m.server_content
if sc and sc.interrupted:
st["interrupted"] = True
if sc and sc.input_transcription and sc.input_transcription.text:
st["transcribed"] = True
if sc and sc.model_turn:
for p in sc.model_turn.parts or []:
if p.inline_data and p.inline_data.data:
st["gen"] = True
st["audio_s"] += len(p.inline_data.data) / 2 / 24000
if sc and sc.generation_complete:
st["gen"] = False
if st["stop"]:
break
async def start_model_turn():
await s.send_client_content(turns=types.Content(
role="user", parts=[types.Part(text=LONG_TURN)]), turn_complete=True)
mic_t = asyncio.create_task(mic())
recv_t = asyncio.create_task(recv())
if user_already_speaking: # ---- case B ----
st["speak"] = True # user is ALREADY talking ...
await asyncio.sleep(1.5)
await start_model_turn() # ... when the model's turn begins
await asyncio.sleep(9) # user keeps talking straight through it
else: # ---- case A ----
await start_model_turn() # model starts talking, user silent
while not st["gen"]: # wait until it is actually generating
await asyncio.sleep(0.05)
await asyncio.sleep(0.6) # ~0.6 s into generation ...
st["speak"] = True # ... user speaks: a FRESH onset
await asyncio.sleep(6)
st["stop"] = True; mic_t.cancel()
try:
await asyncio.wait_for(recv_t, 2)
except Exception:
pass
print(f"{label}: transcribed_user={st['transcribed']} "
f"model_audio={st['audio_s']:.1f}s INTERRUPTED={st['interrupted']}")
return st["interrupted"]
async def main():
a = await run("A fresh onset during generation ", user_already_speaking=False)
await asyncio.sleep(1)
b = await run("B user already speaking at start ", user_already_speaking=True)
print(f"\nExpected: both interrupt the model. Got A={a} B={b}")
print("BUG: case B never emits `interrupted`, though the server transcribes the user's speech.")
asyncio.run(main())
Observed output
A fresh onset during generation : transcribed_user=True model_audio=8.7s INTERRUPTED=True
B user already speaking at start : transcribed_user=True model_audio=23.5s INTERRUPTED=False
Expected: both interrupt the model. Got A=True B=False
BUG: case B never emits `interrupted`, though the server transcribes the user's speech.
Why this matters
For any open-mic / handsfree voice assistant, case B is common: the user starts a sentence, the assistant begins its turn a moment later (e.g. after end-of-turn detection on a prior utterance, or a client_content turn), and now the two overlap. Because no interrupted is emitted — and there is no separate start/stop-of-activity event exposed to the client — the client cannot tell that it should stop, and the assistant talks over the user with no way to yield. Barge-in effectively fails exactly when the user is most insistent.
Notes / ruled out
- Not client-side audio: the same speech interrupts fine (~0.2 s) when its onset lands during generation (case A), and the server returns
inputTranscriptionfor the case-B speech, so the audio is received and recognized. - Not specific to
client_content-initiated turns: aclient_content-forced turn is interrupted normally by a fresh onset. The single deciding variable is onset-during-generation vs. already-speaking-at-turn-start. - Request: either emit
interruptedwhen ongoing user speech overlaps a starting model turn, or expose an explicit start-of-activity / end-of-activity event to clients so barge-in can be handled client-side.