[REST API] Grounding and JSON responses not compatible?

Gemini API
,
1

I am trying to use the REST API to have gemini responses GROUNDED WITH Google Search. I would also like the responses to be in JSON and respect a given JSON schema.
Alas, this seems impossible

  • I have tried gemini-1.5-pro specifiying google_search_retrieval as a tool and I get a 400 error with the following message:
    “Search Grounding can’t be used with JSON/YAML/XML mode.”
  • I have also tried gemini-2.0-flash (using this time google_search as a tool) and I still get a 400 error with a different message:
    "Unable to submit request because Controlled generation is not supported with Google_search tool.. Learn more: Generate content with the Gemini API in Vertex AI  |  Generative AI  |  Google Cloud’ " (but the URL provides zero insight on this specific problem)

IMPORTANT NOTE: the combination of GROUNDED responses and JSON formatting works just fine if i use the python API through the vertexai library
Can someone please shed light on this and hopefully point to a solution?

1 Like
3

I am also seeing the same issue.

5

I have the same problem. Im sure google will say thats because the google search tool has its own response format, the problem though is you cant force it to always use google search on gemini 2 so in the instances where it does not, I need the json response to follow the schema.

7

Google Cloud support’s suggestion for this limitation is to make two requests: one to get the desired response with grounding enabled, and another to take the unstructured response from the first request and structure it according to the desired JSON schema.

This is not a viable solution for me as it will double (or more) the amount I spend on requests to the Vertex API.

9

I would also love if these were compatible. Can’t imagine that not being a very common use case.