## TL;DR
Our Google Cloud Console’s own metrics show `BidiGenerateContent` (the Gemini Live API method powering our real-time voice product) failing **74% of the time**, with **1-minute average / 4-minute 99th-percentile latency**, over the trailing 24 hours — while every other Google API we call on the same project, same network, same timeframe (Google Calendar API) is at **0% errors and 126ms**. This is causing real phone calls to drop, stall for minutes, or silently fail to complete a booking in a live production voice-calling product. We’ve spent significant time ruling out client-side causes (§6) and don’t believe this is something in our code. Full detail, timestamps, and specific call IDs below.
—
## 1. Summary
Two related, likely-connected symptoms on the Gemini Live API (`BidiGenerateContent`), used for real-time voice conversations (Twilio phone calls bridged to Gemini via a native-audio Live session):
1. **Hard session closures** — the Live WebSocket closes mid-conversation with the server-supplied reason: `“The audio content type (CONTENT_TYPE_AUDIO) is not supported for this model configuration.”`
2. **Severe intermittent latency / stalls** — the model goes unresponsive for anywhere from several seconds to multiple minutes mid-conversation, sometimes recovering after the caller prompts again (e.g. “are you there?”), sometimes not.
Google Cloud Console’s own API metrics for this project confirm this is isolated to the Live API specifically, not a general account/network/client issue (see §4).
—
## 2. Environment
| | |
|—|—|
| Model | `gemini-2.5-flash-native-audio-latest` (floating alias, not a pinned dated snapshot) |
| API surface | Live API — `ai.live.connect()` → `google.ai.generativelanguage.v1beta.GenerativeService.BidiGenerateContent` |
| SDK | `@googlegoogle/genai` — observed on both `2.6.0` and `2.15.0` (upgraded mid-investigation specifically to test whether it resolved the issue; it did not) |
| Session config | `systemInstruction` (business-receptionist persona, ~150 words), `tools` (3 function declarations for calendar operations), `inputAudioTranscription: {}`, `outputAudioTranscription: {}`, `speechConfig.voiceConfig.prebuiltVoiceConfig` set. **`responseModalities` deliberately NOT set** (see §5 — removed specifically to test as a fix candidate). |
| Transport | Audio bridged from Twilio Media Streams: caller audio transcoded µ-law 8kHz → PCM16 LE 16kHz before `sendRealtimeInput`; model output PCM16 LE 24kHz → µ-law 8kHz for Twilio. Confirmed via unit tests that this transcoding is real (correct decode/resample math), not mislabeled passthrough. |
| Call volume | Low — this is a pre-launch dev/test account, roughly two dozen real test calls over the incident window, not production client traffic. |
—
## 3. Symptom detail
### 3a. Hard session closure
Consistent server-supplied close reason across every occurrence:
```
The audio content type (CONTENT_TYPE_AUDIO) is not supported for this model configuration.
```
Arrives via the WebSocket **close** event (not an SDK-thrown exception, not our own error handler) — confirmed by our own client-side instrumentation: every occurrence logged through our `onclose` callback, never our separate `onerror` path, which would fire for a client-side/SDK exception instead.
**Representative occurrences** (UTC timestamps, from Twilio call logs cross-referenced with our application logs):
| Call SID | Date/time (UTC) | Duration | Notes |
|—|—|—|—|
| `CAe51c032f1eae8b02611cbb82bb84c5..` | 2026-08-04 ~19:08:55–19:09:47 | 52s | Closed while composing a response involving a scheduling tool call |
| `CA705102585720e096bafe36f50e327a..` | 2026-08-04 19:37:22–19:38:00 | 38s | Closed mid-narration (“I can… certainly help…”) right as a tool call would follow |
| `CAce8e381316f128d95faeb026a2284d..` | 2026-08-04 ~20:58:31–20:58:43 | ~12s of the relevant turn | Closed immediately after “Let me check our availability for tomorrow, August” |
| `CAbcd734b7a3c4fef18fafded1985d27..` | 2026-08-05 13:14:59–13:15:02 | ~3s of the relevant turn | Recurre@google on `@google/genai` 2.15.0, after upgrading from 2.6.0 specifically to test this |
| `CA009e7e7ce645286cfad62d68bc35a8..` | 2026-08-24 17:42:22–17:42:51 | 29s | **Close code captured for the first time: 1007.** Closed immediately after the agent asked “What day works for you?” — a scheduling question about to lead into `getAvailableSlots`. 20 days after this report was first drafted (still never submitted) and 3 days after the calendar.events OAuth verification was approved — confirms this is unrelated to and unaffected by that fix. |
| `CA50a71975424001cedd5977f6e5b60a..` | 2026-08-24 20:35:04–20:36:59 | 115s | Same code 1007 / `CONTENT_TYPE_AUDIO` closure, but this time on a **completely different, longer system instruction** (a newly-added ~6,300-character brand-persona prompt, vs. the shorter home-services one every other occurrence used). Rules out “something specific to the home-services prompt’s length/content” as a cause — the closure reproduces regardless of which system instruction is loaded. |
**Pattern observed across every occurrence, no exceptions:** the closure always happens while the model is generating a spoken response that narrates or leads into a scheduling-related tool call (`getAvailableSlots` / `bookAppointment`) — e.g. “let me check availability,” “I’m booking this for…,” “you’re all set for…”. It has never occurred during plain conversation with no tool call pending or imminent. In every case, our application-level tool-call dispatch logging shows **no evidence the function call was ever sent to us** before the closure — i.e. the failure occurs inside server-side turn generation, before the function-call message is delivered to the client.
Close code was not captured for the 2026-08-04/05 incidents (our logging only recorded the reason string until 2026-08-05). Now captured as of the 2026-08-24 recurrence: **code 1007** (WebSocket protocol error — “invalid frame payload data”), paired with the same `CONTENT_TYPE_AUDIO is not supported` reason string every time.
### 3b. Severe latency / unresponsive stalls
Separately (and possibly related — see §4), the model periodically stops responding mid-call for anywhere from several seconds up to multiple minutes, without any error or close event — the WebSocket stays open, audio continues streaming from our side, but no output arrives until either (a) it eventually resumes on its own, or (b) the caller prompts again, which sometimes appears to “wake” it. Observed same-day alongside the hard closures.
**Recurrence, 2026-08-21** (16 days after this report was drafted, post the 2.6.0→2.15.0 SDK upgrade — confirms this is not resolved): call SID `CA0962b5e5d4f6ee345892f8219ad739..`, started 16:18:24 UTC, 104s total duration. Caller stated their name (triggering the `bookAppointment` tool call); the agent then went silent for ~24 seconds with no audio output. Caller said “are you there?” and the agent responded within ~2 seconds, confirming the booking had in fact completed (a real calendar event was created, `gc5hmlbothi1ri1g8dficvv6e8`). Traced through our own message-handling code (`lib/core/voice/geminiLiveSession.ts`) to rule out a client-side cause: every `serverContent` audio chunk is forwarded immediately with no buffering or state machine that could withhold it — if 24 seconds pass with nothing forwarded, the server genuinely sent nothing during that window. Matches this section’s already-documented pattern exactly, including the “recovers after the caller prompts again” detail.
**New symptom variant observed, 2026-08-21 (same day, later call):** call SID `CAb6795b6f315552cd41daefe727fa6f..`. After a caller stated their name, the agent’s output over the next ~26 seconds was not pure silence but repeated bursts of corrupted transcript chunks — non-printable control characters with no actual text content (the same artifact noted in §3a’s ``-style logging observation, but here occurring as the *entire* content of `outputTranscription` events, not just a display quirk around otherwise-real content). This suggests the stall and the garbled-output artifact may be the same underlying server-side issue manifesting two ways, rather than two separate bugs. Pulled directly from Railway’s runtime logs (Railway GraphQL API, `deploymentLogs` query) rather than inferred — full per-millisecond timeline available if useful to Google’s team.
**Confirmed audible, 2026-08-24 (call SID `CAbfcb196533191a25fefa1ed53999ee..`):** same garbled-chunk pattern recurred, and this time the person on the call could hear it directly rather than us only inferring it from logs — described as the agent’s speech sounding “semi garbled for like 2 seconds” before recovering cleanly. Confirms the corrupted `outputTranscription` chunks correspond to genuinely corrupted *audio* reaching the caller, not just a transcript/logging-layer artifact with unaffected audio underneath.
—
## 4. Quantitative evidence (Google Cloud Console, this project’s own metrics)
From **APIs & Services → Gemini API**, trailing ~24h window ending 2026-08-05 afternoon:
| Method | Requests | Error rate | Avg latency | 99th percentile latency |
|—|—|—|—|—|
| `GenerativeService.BidiGenerateContent` | 27 | **74.07%** | **1 minute** | **4 minutes** |
| `ModelService.GetModel` | 1 | 100% | 0.049s | 0.065s |
| `ModelService.ListModels` | 1 | 100% | 0.037s | 0.065s |
For comparison, **Google Calendar API** (a different Google API, same project, same network, same time window, called by the same application): 45 requests, **0% errors**, 126ms median latency, 458ms 95th-percentile latency.
Notes on this table:
- The two `GetModel`/`ListModels` errors are our own diagnostic calls made with a since-identified stale/invalid local API key (unrelated to production traffic) — both fail in under 50ms, consistent with an instant auth rejection. They cannot explain the `BidiGenerateContent` numbers.
- The `BidiGenerateContent` average latency of a full minute (up to 4 minutes at p99) **cannot** be explained by an invalid-key rejection (which fails instantly, as the other two rows demonstrate) — this reflects real, working-credential traffic taking an abnormal amount of time.
- The stark contrast with Calendar API (0% errors, 126ms) on the same account, network, and timeframe rules out a general connectivity/environment/client-side network issue.
—
## 5. Possibly-related Google-side billing incident (uncertain — flagging, not claiming)
While investigating, we found a confirmed, active Google Cloud Service Health incident on this project’s billing account:
- **Event ID:** `JZSMLXZ` — “Cloud Billing customers may experience delays in receiving new cost data updates and BigQuery exports.”
- **Status:** Active (Confirmed) at time of writing.
- **Timeline:** Incident start 2026-08-02 08:02:57 AM UTC-4; symptom identified 2026-08-03 4:35:05 PM UTC-4; per the most recent update, “mitigated as of Tuesday, 2026-08-04 16:02 PDT,” with historical cost data since Aug 1 still backfilling.
- **Google’s own relevance assessment for this project:** “Partially related” — “the incident may not be impacting your project.”
- We separately encountered a “write access to project denied, please check billing account associated and retry” error in Cloud Console around the same window, which may be a symptom of this same incident rather than a distinct misconfiguration on our end.
**We are not claiming this is the cause** of the `BidiGenerateContent` errors/latency in §3–4 — the timing doesn’t cleanly support it: this incident’s stated scope is cost/usage *data reporting* (not API request handling), and it was reported mitigated *before* several of our worst `BidiGenerateContent` measurements (including the §4 table, pulled 2026-08-05, after the stated mitigation time). Flagging it in case Google’s support team recognizes a shared underlying cause we wouldn’t be able to see from the outside.
—
## 6. What we’ve already ruled out client-side
Before concluding this is server-side, we audited our own implementation against six hypotheses:
1. **Model configuration** — exactly one model string used throughout (`gemini-2.5-flash-native-audio-latest`), consistent across all config sources. Confirmed using the Live API (`ai.live.connect`), not `generateContent`. `responseModalities` explicitly **not** set (removed as a fix candidate after finding an unrelated report of a similar symptom tied to that parameter being set on a native-audio model, per SDK docs that show it used only with half-cascade models) — issue persisted with and without it.
2. **Function-call/tool-response boundary** — tool results are returned via the SDK’s dedicated `sendToolResponse`/`functionResponses` structure (never as audio or generic content), with correct `id` correlation back to the original function-call id. No session recreation occurs between a function call and its response.
3. **Shared conversation history** — the Gemini client is instantiated in exactly one place in our codebase; there is no separate text-only model, no RAG-into-Live-session integration (RAG exists but isn’t wired into the live call path at all yet), and no shared history array anything could leak through.
4. **Audio format** — verified the actual transcode implementation (not just labels): real µ-law decode/encode, real resampling 8kHz↔16kHz/24kHz, MIME type (`audio/pcm;rate=16000`) matches the actual sample rate sent. Backed by unit tests including round-trip fidelity checks.
5. **SDK version** — upgraded `@google/genai` 2.6.0 → 2.15.0 specifically to test whether this resolved it. It did not; identical symptom on both versions.
6. **Client-side event ordering** — audio continues streaming from our side continuously and is never gated on tool-call state, but our logs show the failure occurs *before* we ever receive the function-call message from the server — i.e. before any client-side ordering choice on our part could be relevant.
—
## 7. Ask
1. Is there a known issue with `BidiGenerateContent` returning `CONTENT_TYPE_AUDIO is not supported for this model configuration` when a Live session with function calling enabled generates a response that includes (or leads into) a tool call?
2. Is there a known capacity/latency issue affecting `gemini-2.5-flash-native-audio-latest` Live sessions in the relevant timeframe/region that would produce the latency profile in §4?
3. Is this project’s API key on a quota/tier that could explain the elevated error rate and latency, and would enabling billing / upgrading tier be expected to resolve it?
4. Are `goAway` / `sessionResumptionUpdate` messages being sent before these closures? (We’ve only just added client-side logging for these signals and have not yet observed one, but wanted to flag we’re specifically watching for this as a possible legitimate session-duration-limit interaction we may not be handling correctly.)
5. Is Cloud Billing incident `JZSMLXZ` (§5) connected to `BidiGenerateContent` reliability in any way, even indirectly (e.g. shared quota/usage-tracking infrastructure)?
Happy to provide full request logs, additional call SIDs, or a minimal reproduction if useful.