Hey team,
Wanted to flag something we’re seeing with Gemini 2.0 Flash, specifically around tool usage in longer conversations.
We’ve noticed that the longer the chat goes on, the more hesitant - or just plain unwilling - the model gets about actually calling the 7 tools we’ve set up. We’re using the latest Vercel SDK, and the prompt was even built specifically for this model.
The weird thing is, even when we try to explicitly tell the model in the prompt or query to use a tool, it often just skips it after a certain number of turns.
What’s really throwing us is that we took the exact same prompt and setup and tested it with a competitor’s model, and it worked perfectly – consistently using the tools no matter how long the conversation got.
So, it feels like it’s something specific to how Gemini 2.0 Flash handles tool calling when the context grows.
Has anyone else run into this with 2.0 Flash and tools? Or have any ideas on what might be going on? It’s getting a bit frustrating for our use case.