Grounded Search not working with Gemini 1.5 PRO (works on console not on API)

Gemini API
, ,
1

For our use case, we need a model to process long text and be able to reason.

Gemini 2.0 Flash - Grounded search works, but model output is unstable and we see frequent hallucinations.

Gemini 2.0 Pro Experimental - Its experimental model, cant put into production.

Gemini 1.5 Pro - The best model for our use case. Its stable with great quality outputs. However starting 10 days back, the grounded search functionality is not working in API.

Can anyone in Google team look into this? I am aware of the plan to stop Gemini 1.5 Pro. But until the Gemini 2.0 Pro is production ready, can all the features be made available at 1.5 Pro?

2

Hi @Subu_Ramachandran, Could you please elaborate more on this. not working means, are you getting any error or the model response is not good etc. Thank You.

5

Hi @Kiran_Sai_Ramineni , For 1.5-pro, we used Tool google_search_retrieval option for query’s “tools” option. The error was API complained about input tokens being more than 32k, while we know it was not the case (7k token prompt). With 2.0-flash, the problem went away. However, Gemini API requires the Tool being slightly different: google_search with types.GoogleSearchRetrieval as value. We are using chat feature.

In both cases DynamicRetrievalConfig was applied. Although 2.0-flash works in this case, it frequently outputs texts that are not JSON compliant (because Gemini does not support web grounding with JSON schema, we tell Gemini to generate JSON in our prompt. 1.5-pro does a good job here). In addition, results from 2.0-flash had hallucinations from time to time when web grounding was applied.