Use JSON output with Gemini gemini-1.5-pro-001

Gemini API
1

If I try to use:
generation_config=GenerationConfig(
response_mime_type=“application/json”,
max_output_tokens=8192,temperature = 0.1,
top_p=0.95,
response_schema =schema)

and the schema is an object schema, then get :
slight_smile: Traceback (most recent call last):
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/rules/message.py”, line 36, in to_proto
return self._descriptor(**value)
^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Parameter to CopyFrom() must be instance of same class: expected <class ‘Schema’> got <class ‘dict’>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/message.py”, line 570, in init
pb_value = marshal.to_proto(pb_type, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/marshal.py”, line 226, in to_proto
return {k: self.to_proto(recursive_type, v) for k, v in value.items()}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/marshal.py”, line 228, in to_proto
pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/rules/message.py”, line 36, in to_proto
return self._descriptor(**value)
^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Protocol message Schema has no “$ref” field.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/message.py”, line 570, in init
pb_value = marshal.to_proto(pb_type, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/marshal.py”, line 226, in to_proto
return {k: self.to_proto(recursive_type, v) for k, v in value.items()}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/marshal.py”, line 228, in to_proto
pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/rules/message.py”, line 41, in to_proto
return self._wrapper(value)._pb
^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/message.py”, line 598, in init
pb_value = marshal.to_proto(pb_type, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/marshal.py”, line 226, in to_proto
return {k: self.to_proto(recursive_type, v) for k, v in value.items()}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/marshal.py”, line 228, in to_proto
pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/rules/message.py”, line 36, in to_proto
return self._descriptor(**value)
^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Protocol message Schema has no “$ref” field.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/rules/message.py”, line 36, in to_proto
return self._descriptor(**value)
^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Parameter to CopyFrom() must be instance of same class: expected <class ‘Schema’> got <class ‘dict’>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/message.py”, line 570, in init
pb_value = marshal.to_proto(pb_type, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/marshal.py”, line 226, in to_proto
return {k: self.to_proto(recursive_type, v) for k, v in value.items()}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/marshal.py”, line 228, in to_proto
pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/rules/message.py”, line 36, in to_proto
return self._descriptor(**value)
^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Protocol message Schema has no “$ref” field.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/main.py”, line 514, in
generation_config=GenerationConfig(
^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/vertexai/generative_models/_generative_models.py”, line 1360, in init
raw_schema = aiplatform_types.Schema(gapic_schema_dict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/message.py”, line 598, in init
pb_value = marshal.to_proto(pb_type, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/marshal.py”, line 226, in to_proto
return {k: self.to_proto(recursive_type, v) for k, v in value.items()}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/marshal.py”, line 228, in to_proto
pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/rules/message.py”, line 41, in to_proto
return self._wrapper(value)._pb
^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/message.py”, line 598, in init
pb_value = marshal.to_proto(pb_type, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/marshal.py”, line 226, in to_proto
return {k: self.to_proto(recursive_type, v) for k, v in value.items()}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/marshal.py”, line 228, in to_proto
pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/proto/marshal/rules/message.py”, line 36, in to_proto
return self._descriptor(**value)
^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Protocol message Schema has no “$ref” field.

If I try to use
class schema (TypeDict):

Instead of defining a hierarchical output class for JSON then, I’m getting:
Traceback (most recent call last):

File “/main.py”, line 514, in
generation_config=GenerationConfig(
^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/vertexai/generative_models/_generative_models.py”, line 1359, in init
gapic_schema_dict = _convert_schema_dict_to_gapic(response_schema)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/vertexai/generative_models/_generative_models.py”, line 1677, in _convert_schema_dict_to_gapic
if “type” in gapic_schema_dict:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument of type ‘_TypedDictMeta’ is not iterable

1 Like
2

from vertexai.generative_models import TextGenerationModel, GenerationConfig
import aiplatform_types

Define your Text Generation model

model = TextGenerationModel.from_pretrained(“text-davinci-003”)

Define your schema using aiplatform_types.Schema

schema = aiplatform_types.Schema({
“type”: “object”,
“properties”: {
“name”: {“type”: “string”},
“age”: {“type”: “integer”},
“city”: {“type”: “string”}
}
})

Generate text with JSON output using the schema

generation_config = GenerationConfig(
response_mime_type=“application/json”,
max_output_tokens=8192,
temperature=0.1,
top_p=0.95,
response_schema=schema
)

response = model.predict(
“Please generate a JSON object with the following information: name, age, and city.”,
generation_config=generation_config
)

Access the generated JSON data

json_output = response.results[0].text
print(json_output)

3

I have been using Open AI JSON since January, and we already have an app in production.
We were trying to test Gemini 1.5 Pro and Flash. Gemini Pro and Flash use different approaches to JSON. The flash model works with the schema in the prompt, and we have run a few successful tests so far. Gemini Pro assumes using a typing schema approach according to published examples. We try to use the example for our schema that has iterations with custom nested classes and lists, and that one is failing. We want to understand how to create predictable JSON generation and Gemini validation that is comparable to the consistency we got from the Open AI GPT models.

4

Welcome to the forum. It is refreshing to have a poster complain about Google API exhibiting non-compliance with specifications. Notice the definition for Schema: https://ai.google.dev/api/rest/v1beta/Schema

It states “Represents a select subset of an OpenAPI 3.0 schema object.”

The $ref syntax is one of various JSON Schema syntax elements not supported by the Google implementation, so all the $ref raised Exceptions in your first attempt.

Unfortunately, Google doesn’t elaborate on the specific areas where their implementation diverges from the specification it attempts to adhere to. Users of the API are left with trying it out and determining for themselves what works and what doesn’t.

Hope that helps!