I’ve encountered an issue with the Gemini API where there seems to be an undocumented size limit for the response_schema parameter in GenerationConfig. When attempting to use a schema with a large number of properties or long property names, the API throws an InvalidArgument error. What’s worse, there’s no error description.
This behavior isn’t mentioned in the documentation, and it’s unclear what the exact limits are. Has anyone else experienced this? Are there any official guidelines on the maximum size or complexity of the response_schema?
I’ve attached a minimal reproducible example demonstrating the issue.
I encourage you to try different parameter values. For example:
With these parameters the request succeeds consistently:
num_properties = 42
property_name_length = 34
However, these consistently causes an InvalidArgument exception:
num_properties = 43
property_name_length = 34
And so do these:
num_properties = 42
property_name_length = 35
This leads me to believe that the combination of combined property name lengths could be the issue.
I’m looking forward to someone looking behind the GAPIC veil
import copy
from pprint import pprint
import random
import string
from vertexai.generative_models import GenerationConfig, GenerativeModel
from google.api_core.exceptions import InvalidArgument
prompt = "Respond according to the JSON schema."
num_properties = 40
property_name_length = 40
properties = [
"".join(random.choices(string.ascii_lowercase, k=property_name_length))
for _ in range(num_properties)
]
json_schema = {
"type": "object",
"properties": {name: {"type": "string"} for name in properties},
}
model = GenerativeModel("gemini-1.5-pro-001")
try:
response = await model.generate_content_async(
contents=prompt,
generation_config=GenerationConfig(
temperature=0.0,
response_mime_type="application/json",
response_schema=copy.deepcopy(json_schema),
),
stream=True,
)
except InvalidArgument as e:
print("Request failed as expected with InvalidArgument error:")
print(e)
print(
f"generation_config.response_schema had {num_properties} properties, {property_name_length} characters each:"
)
pprint(json_schema)
Interestingly, I’ve noticed that I can pass a substantially larger request schema to the API on VertexAI, but with AI Studio it fails with a much smaller schema
