Gemini 3.8 Flash Agentic Video: HTTP 200 then api_error after 23 processing calls

Gemini API
1

I’m seeing a reproducible failure with Agentic Video Understanding through the Gemini Developer API’s Interactions API. This is not Vertex AI or Batch API. It blocks evaluation of a video-timeline analysis workflow.

Observed behavior

  1. Upload the MP4 through Files API and wait until it is ACTIVE.
  2. Call client.interactions.create(..., stream=True) through the official Python SDK.
  3. The generation POST returns HTTP 200 and streaming begins.
  4. The stream emits 23 processing_call steps and 22 completed processing_result steps, then:
{
  "code": "api_error",
  "message": "Internal error encountered."
}

No final response JSON or usage metadata arrives. This is an error inside an already-open HTTP 200 stream, not an HTTP 500 response to the initial POST. Each attempt makes one generation POST, with no automatic generation retries.

Environment and request shape

  • Model: gemini-3.8-flash (availability checked before submission)
  • SDK: google-genai==2.22.0
  • Python 3.12.13 on Fedora
  • Endpoint: https://generativelanguage.googleapis.com/v1beta/interactions
  • Standard streaming requests, agentic video processing, low media resolution

Sanitized request outline below; prompt and schema represent the actual task instructions and JSON schema, not literal placeholders sent to the API:

stream = client.interactions.create(
    model="gemini-3.8-flash",
    input=[
        {
            "type": "video",
            "uri": uploaded_file.uri,
            "mime_type": "video/mp4",
            "processing": "agentic",
            "resolution": "low",
        },
        {"type": "text", "text": prompt},
    ],
    response_format={
        "type": "text",
        "mime_type": "application/json",
        "schema": schema,
    },
    stream=True,
)

The task asks for WATCH/SKIP state transitions and natural scene breaks in a complete video. The schema is an object with two required arrays, transitions and scene_breaks. Entries contain non-negative integer timestamp_ms, a low|medium|high confidence enum, and (for transitions) a WATCH|SKIP state enum. It uses $defs/$ref; scene_breaks has maxItems: 6.

Reproduction and controls

  • 7/7 different full-length files failed with the same 23-call/22-result signature. They span approximately 88–147 minutes and different resolutions. Generation time was approximately 156–304 seconds.
  • These exact local media files previously produced successful answers using an earlier prompt/schema and request configuration. That historical comparison is not prompt-only: several envelope settings also differed.
  • On one file, I retained the current prompt/schema/media and minimal SDK path, then added only the following generation configuration from an earlier successful run:
{
  "max_output_tokens": 8192,
  "seed": 20260904,
  "thinking_level": "low",
  "thinking_summaries": "none"
}

That controlled attempt also failed at 23/22 after approximately 963 seconds of generation.

  • A separately timestamp-corrected, decoded-frame-equivalent media derivative also reproduced the failure. This rules out the observed local timestamp defect as necessary for that failure; it does not identify the provider-side cause.
  • Earlier successful runs exceeded 23 processing calls, so I am not claiming a universal 23-call ceiling.
  • The error did not contain an explicit safety, copyright, or recitation-block diagnosis. Missing usage means cost is unknown, not zero.

Questions for the Gemini team

  1. Is this repeated unfinished processing-step pattern a known Agentic Video issue? Could it reflect an internal navigation/tool failure, timeout, or structured-output interaction?
  2. Is the request shape above correct for long-form agentic video with structured JSON? Are there relevant restrictions on $defs/$ref, maxItems, or generation settings in this combination?
  3. Which diagnostic identifiers/events should I retain, and is there a private channel where staff can inspect interaction IDs and sanitized receipts? I have intentionally excluded identifiers and media from this public post.
  4. Can failed interactions be billed when no usage metadata is returned, and how can those charges be reconciled?

I can provide further sanitized prompt/schema details. I am looking for a way to diagnose the underlying failure rather than repeatedly resubmitting paid requests.

2

Update: two subsequent non-streaming attempts returned an explicit HTTP 400 “too many tool calls” error, rather than the earlier HTTP 200/SSE api_error. All timestamps below are UTC and refer to when the error response was received.

Attempt Error received (UTC) Inference elapsed Result
First explicit tool-call error 2026-09-07 21:06:35 127.356 s HTTP 400
Identical-request repeat 2026-09-08 04:02:49 136.387 s HTTP 400

Both returned this error body:

{
  "error": {
    "code": "Model generated function call(s).",
    "message": "Model generated too many tool calls. Please retry the request. If the issue persists, include this error message in the retry prompt to allow the model to call a valid number of tools."
  }
}

Settings when this started appearing

Still the official google-genai==2.22.0 SDK, gemini-3.8-flash, Interactions API, and the same complete MP4. The relevant settings were:

service_tier="standard"
stream=False
background=False
store=False
generation_config={
    "thinking_level": "low",
    "thinking_summaries": "auto",
}
response_format={"type": "text", "mime_type": "application/json"}
# Video input: processing="agentic", resolution="medium"
# No explicit max_output_tokens, seed, previous_interaction_id,
# tools array, or custom API-revision header.

There was no schema in response_format for these two calls: JSON MIME mode only, with the field definitions in the prompt. Local schema validation would run only after a response was returned. The prompt also used total file duration for its timing rules instead of calculating accumulated WATCH duration.

Immediately before the first Standard attempt, I tried the same request on Flex. That returned HTTP 500/api_error with an explicit high-demand message. I then changed only service_tier from Flex to Standard and received the first HTTP 400 above. The second Standard attempt had the exact same request-body SHA-256 as the first. Both already used Low thinking.

This is not a one-variable comparison against the original streaming report: stream mode, resolution, response format and other settings had changed along the way. I am reporting when the explicit error first appeared, not claiming which change exposed it.

What is and is not observable

  • One generation POST per Standard attempt; no automatic retry or new upload. The existing file was ACTIVE before and after both attempts and was not near expiry.
  • Neither returned output JSON, usage metadata, an interaction ID, a request-ID header, or a processing trace. I cannot assign a 23-call count to these non-streaming failures.
  • Flattening the schema and increasing the output cap to 65,536 had already failed to resolve the earlier streaming error. A separate Medium/auto-summaries streaming attempt closed all 23 processing results before the generic error, so the unfinished-result pattern is not universal either.
  • The earlier successful run with 26 matched processing pairs still prevents a claim of a universal 23-call ceiling. Missing usage still means unknown billing, not zero cost.

Could the Gemini team clarify whether Agentic Video has a per-request limit on tool calls, agent turns, or tool-result volume; whether it is fixed or dynamic; and whether the earlier streaming api_error can be the same condition surfaced differently? Is there a supported way to observe or configure that budget, or retain the failed internal tool details? I can share diagnostic identifiers privately with staff.