Gemini-3.8-flash ignores responseJsonSchema in Batch API (inline requests); same request is schema-valid synchronously

,

Model: gemini-3.8-flash (Gemini Developer API, Batch mode, inline requests)
SDK: google-genai 3.2.0 (Python)

Summary

In Batch mode, gemini-3.8-flash returns JSON that does not follow responseJsonSchema: required properties are missing, objects come back as strings or null, extra keys appear. The batch job ends JOB_STATE_SUCCEEDED and every response has finishReason: STOP, so nothing signals the failure, and the tokens are billed.

The identical request sent synchronously (generateContent / streamGenerateContent) returns schema-valid output every time. The same code path in Batch mode with gemini-4-carbon is also schema-valid.

Numbers

Mode Model Schema-valid responses
Batch, inline gemini-3.8-flash 0 / 123 (job batches/l70vw2m4fhgfqu0yit1gnljci3bdc9wt314y)
Batch, inline (probe) gemini-3.8-flash 0 / 6 (job batches/pc7b06jatqcatjns5pfcgi3vprglhag080ux)
Sync gemini-3.8-flash 22 / 22
Batch, inline gemini-4-carbon all valid, several jobs

Of the 123, 7 ended with MAX_TOKENS (truncation is a separate issue). The other 116 ended with STOP but were off-schema.

What the probe ruled out

The probe sent each of 2 prompts with 3 configs in one batch. All 6 responses were off-schema:

  • A: responseJsonSchema + thinkingConfig {includeThoughts: true, thinkingLevel: HIGH}
  • B: responseJsonSchema, no thinkingConfig
  • C: responseJsonSchema + thinkingConfig {thinkingLevel: HIGH} (no thought summaries)

So thinking settings do not matter. The serialized inline request does contain generationConfig.responseMimeType: "application/json" and generationConfig.responseJsonSchema. The output is JSON, so the MIME type is honoured. The schema is not.

Schema (abridged)

{
  "type": "object",
  "properties": {
    "summary": {"type": "string"},
    "edits": {"type": "array", "items": {"type": "object",
      "properties": {"quote": {"type": "string"}, "issue": {"type": "string"}, "suggestion": {"type": "string"}},
      "required": ["quote", "issue", "suggestion"], "additionalProperties": false}},
    "keep": {"type": "array", "items": {"type": "string"}},
    "scores": {"type": "object",
      "properties": {"emotion": {"type": "integer", "enum": [1,2,3,4,5,6,7,8,9,10]}, "...": "6 more like this"},
      "required": ["emotion", "..."], "additionalProperties": false},
    "verdict": {"type": "string", "enum": ["publish", "minor_revision", "major_revision", "reject"]}
  },
  "required": ["summary", "edits", "keep", "scores", "verdict"],
  "additionalProperties": false
}

Typical violations in the batch output

  • edits[i] is a string instead of an object
  • scores or edits missing
  • scores is not an object, or is null
  • keep missing
  • in some cases degenerate output with one key repeated many times

Repro

from google import genai
from google.genai import types

client = genai.Client()
cfg = types.GenerateContentConfig(
    system_instruction="You are a literary critic. Review the text.",
    response_mime_type="application/json",
    response_json_schema=SCHEMA,   # the schema above
    max_output_tokens=32000,
    thinking_config=types.ThinkingConfig(thinking_level="HIGH"),
)
req = types.InlinedRequest(contents="<a ~20-line text>", metadata={"key": "1"}, config=cfg)
job = client.batches.create(model="gemini-3.8-flash", src=[req])
# poll until JOB_STATE_SUCCEEDED, then validate
# job.dest.inlined_responses[0].response.text against SCHEMA with jsonschema: fails.
# client.models.generate_content(model="gemini-3.8-flash", contents=..., config=cfg): passes.

Expected

Batch responses are constrained by responseJsonSchema the same way synchronous responses are. If the model does not support constrained decoding in Batch mode, the request should be rejected at submission.

Related