support caching response_schema/structured-output config in CreateCachedContentConfig

Gemini API
, ,
1

CreateCachedContentConfig currently accepts system_instruction, contents, tools, and tool_config, but not response_schema/response_mime_type. For prompts with large structured-output schemas, this means the schema is resent at full price on every call even when system_instruction is cached.

The only current workaround is converting the schema into a single forced-choice FunctionDeclaration/Tool and baking that + tool_config into the cache instead — which we validated works (schema tokens do show up under cached_content_token_count), but it comes with real downsides vs. native response_schema support:

  • Loses whatever constrained-decoding guarantee response_schema mode provides — function-calling isn’t documented as giving the same schema-adherence guarantee.
  • Introduces MALFORMED_FUNCTION_CALL as a new failure mode (see #1789) with no equivalent under response_schema mode.
  • Requires a full generate_content config swap (can’t mix response_schema with tools/tool_config in the same cached request per current API validation), so cache-hit vs. cache-miss calls need materially different request shapes.

Feature request: allow response_schema/response_mime_type to be part of the cached content directly, so structured-output prompts get the same caching benefit as system-instruction-heavy ones without the function-calling workaround.

2

Hello @dexter1 ,

Thank you for the feedback, we have created a feature request for this internally for the team.

3

Hi @dexter1 ,
I’m the PM on the Gemini API team focused on Explicit Caching.
I received the feature request and would like to learn more about your use-case and requirements and discuss what we are currently have on the roadmap. If you are open to chat, please feel free to reach out to me directly.

4

Hi,

I’ve also ran into this issue. My prompt is about 6k tokens and my response schema is about 2k tokens. 1k tokens for the rest. When I use implicit caching, I reach about 8k tokens cached tokens. When I use explicit caching (without the response schema), I reach about 6k tokens cached. This is due to response schema not being cached.

This effectively means that explicit caching is unusable for my purposes. My requests end up costing ~50% more with explicit caching.

What I have in the response schema is essentially rules for how how to structure the output in json format. I have quite a long list of fields with instructions.

I tried moving the response schema into the tool fields, but the results of my queries changed and were no longer as stable as before.

Johnny