Unexpected InvalidArgument error for large response_schema

Gemini API
1

I’ve encountered an issue with the Gemini API where there seems to be an undocumented size limit for the response_schema parameter in GenerationConfig. When attempting to use a schema with a large number of properties or long property names, the API throws an InvalidArgument error. What’s worse, there’s no error description.

This behavior isn’t mentioned in the documentation, and it’s unclear what the exact limits are. Has anyone else experienced this? Are there any official guidelines on the maximum size or complexity of the response_schema?

I’ve attached a minimal reproducible example demonstrating the issue.

I encourage you to try different parameter values. For example:

  • With these parameters the request succeeds consistently:
    • num_properties = 42
    • property_name_length = 34
  • However, these consistently causes an InvalidArgument exception:
    • num_properties = 43
    • property_name_length = 34
  • And so do these:
    • num_properties = 42
    • property_name_length = 35

This leads me to believe that the combination of combined property name lengths could be the issue.

I’m looking forward to someone looking behind the GAPIC veil :ghost:

import copy
from pprint import pprint
import random
import string
from vertexai.generative_models import GenerationConfig, GenerativeModel
from google.api_core.exceptions import InvalidArgument


prompt = "Respond according to the JSON schema."
num_properties = 40
property_name_length = 40

properties = [
    "".join(random.choices(string.ascii_lowercase, k=property_name_length))
    for _ in range(num_properties)
]

json_schema = {
    "type": "object",
    "properties": {name: {"type": "string"} for name in properties},
}

model = GenerativeModel("gemini-1.5-pro-001")

try:
    response = await model.generate_content_async(
        contents=prompt,
        generation_config=GenerationConfig(
            temperature=0.0,
            response_mime_type="application/json",
            response_schema=copy.deepcopy(json_schema),
        ),
        stream=True,
    )
except InvalidArgument as e:
    print("Request failed as expected with InvalidArgument error:")
    print(e)
    print(
        f"generation_config.response_schema had {num_properties} properties, {property_name_length} characters each:"
    )
    pprint(json_schema)
2 Likes
2

Encountering the same issue here.

3

Happening to me as well! Here’s more duplicates:

https://www.googlecloudcommunity.com/gc/AI-ML/Unexpected-400-errors-with-Generated-Output-Schema/td-p/807575

4

Interestingly, I’ve noticed that I can pass a substantially larger request schema to the API on VertexAI, but with AI Studio it fails with a much smaller schema

6

I noticed your reply in the other thread and re-ran my example snippet above. Unfortunately it’s still returning the same exception.

Would you mind sharing an example that’s working for you, or trying to run the example above in your account?