Gemini responds with structured / JSON-like output only when Function Calling is enabled

I’m facing this issue as well in an agentic app. For me, it occurs when using Gemini 3 Flash; I couldn’t reproduce it with the 2.5 Flash or 2.5 Flash Lite (September preview) models.

I’ve tested both the model provided by Vertex AI and the one from Google AI Studio. The AI Studio API version always fails, returning nonsensical JSON. Vertex AI has a higher success rate, but it often includes a section starting with pasted_thought that exposes the model’s raw chain-of-thought (which is different from the thought summaries).

Note: I’m using LiteLLM as a proxy.

Also happening to us with Gemini3 Pro or Flash

It started yesterday. It is interesting because if you ask it to answer with “PURE TEXT” it will do, but the answer will come inside a string. To be fair, it is unusable currently

4 Likes

Please has this been resolved ? i am having the same issue

1 Like

This is happening for me as well, unusable now. Whether direct API call or via Vercel AI SDK. Something is broken internally in the model!

1 Like

We also stumbled across this issue using the OpenAI style endpoint for Gemini 3 models with tools. Since roughly a day, we get a Json string in the message content instead of just a normal string.

Would like to have that resolved. For now the fallback is to go back to 2.5 flash instead of 3 flash.

1 Like

Same happening for me with gemini 3 pro.

But as well have infinite loop. It keeps returning same text when in a loop with AI SDK. And can not finish

The same is happening for me.

Both Gemini 3 Pro and Flash behave the same way.

I am using the Vercel AI SDK, I didn’t change anything in the code recently, and suddenly, Gemini 3 models started behaving strangely.

ai@5.0.113
@ai-sdk/google@2.0.46

On Vercel AI SDK, this solved it for me for now.

1 Like

I’m seeing this as well on gemini flash 2.5 and 3.0. Using the HTTP API. Regression seems to have started yesterday (Dec 22). All responses are coming back as JSON now (not using structured output, setting functionCallingConfig.mode to AUTO)

thanks for sharing same fix is working for me on genai, as long as i don’t set tool_config to AUTO it works again

1 Like

I have tried @khbiotech ‘s approach,

the issues is still going, i removed AUTO and sometimes it sends text parts that look like tool calls without the tool name and IT NEVER produces a text part if there is a tool call, there is no reason for that, a single turn can contain both text parts and tool calls.

Did you talk to the itnernal team @Srikanta_K_N ??? ha?

1 Like

Good news! Here is some workaround that works in the ai-sdk. You must apply a patch.

Here is the GitHub issue discussion: [Bug] Gemini 3 Pro/Flash Preview models output internal JSON as text when tools are provided · Issue #11396 · vercel/ai · GitHub

Here is an example commit you implement with update dependencies + patch: feat(ai): upgrade AI SDK to v6 with Gemini 3 toolConfig patch · levante-hub/levante@fb67e5b · GitHub

The provided work arounds (not setting AUTO) dont fix the problem, the model still tries to call tools using text parts (using latest java genai sdk)

Same problem here, with flash3 and gemini3 pro. I am in a refactoring process, so you can imagine that I thought the problem was with my code. I will change to Anthropic for now; I hope we get a fix on this.

Hi, we are in a refactoring process, you can imagine we thought our code was the problem, especially after I checked out the API status and it says that everything works ok.

Why don’t you update the API status? If it doesn’t work, it’s not normal to have it on green.

Hey folks, we are looking into this on our side. If anyone has a minimum reproducible code sample that would help. I have not been able to repro yet.

It does sounds like this is happening across both the Google SDK’s and the Vercel AI SDK so potentially no SDK release related but trying to track that down too. Hang tight!

1 Like

Logan in case it helps there is a big thread here as well where someone found a temp patch for now: [Bug] Gemini 3 Pro/Flash Preview models output internal JSON as text when tools are provided · Issue #11396 · vercel/ai · GitHub

1 Like

Okay team has reproduced, working to identify the change that caused this and roll it back.

5 Likes

Com on Logan, you can tell us, we wont tell anybody