Problems With gemini-2.0-flash Tool Calling

Gemini API
,
1

Greetings, everyone. I’m working on a project that utilizes the Vercel AI SDK, which has been very stable with tool calling using GPT-4o and the new Gemini 2 Pro experimental model. However, with Gemini-2.0-flash, I’m encountering issues where, after the message thread includes several tools cals, and particularly the same repeated tool call, the flash model starts to output markdown python code (see below) instead of actually calling the tool. Reprompting it sometimes makes it call the tool, but the issue tends to worsen as the thread grows. I’m trying to determine if this is a problem with the model itself or if it’s an issue with accessing GenAI via the Vercel SDK, as opposed to using the Google Node library. If you are experiencing this with other libraries or have insights on how to resolve this, your input would be greatly appreciated.

Example Response
I have opened the homepage in a new window. Now I need to extract all URLs in the navigation at the top. I will get the HTML of the page.

print(default_api.getBrowswerWindowHtml(BrowserWindowResource = "4f9608a5-55c8-457d-854d-1292b9b44010"))
4 Likes
3

I’m having the same issue. It’s not related to Vercel AI SDK, since I’m just making HTTP requests to the API and getting the same response sometimes. It looks like a model’s issue. Have you found a workaround for this?

2 Likes
4

We see the same thing. Temporary work around we are using is to filter these out using regexp. String.replace(response, ~r/tool[\w]*[\s\S]*?/m, “”) this is elixir code. but you get the idea

3 Likes
5

Thanks for the insight on this.

1 Like
8

Hey folks, thanks for flagging this! Just to confirm, what models are you seeing this behavior with? Is it mainly with 2.0 Flash, or with other models, too?

1 Like
10

I have really only encountered it on flash 2.0, 1.5 pro seems fine and I wasn’t have issues with 2.0 pro experimental though I have not used it heavliy.

2 Likes