I’m having a lot of trouble with the Gemini API’s function calling. I’m trying to build a pretty simple app to get my university class schedule, and the model keeps hallucinating, even with the most basic requests. It’s not a complicated task; it’s just fetching data based on clear inputs.
I’ve set up functions like get_class_schedule(class_names) and get_professor_schedule(professor_name). I’ve provided detailed descriptions, the required parameters, and even the exact output format in the system instructions. I’m using the latest SDK, setting temperature to 0, and I’m providing a well-structured conversation history.
But, the model frequently:
Ignores Function Calls: It responds with generic, made-up text, completely ignoring the available functions.
Inconsistent Behavior: Sometimes it works, other times it hallucinates. And the worst part is, it’s totally unpredictable. One minute it’s working, the next it’s completely off. I can’t figure out a pattern!
I’ve spent a lot of time tweaking prompts, refining system instructions, and trying different settings. I’ve followed the best practices in the documentation, The result? It’s consistently inconsistent, and it’s really frustrating.
We’ve been struggling through this ourselves. We’ve found that the system instructions are incredibly important. Without the right instructions we’ve seen repeated failure. The descriptions in the function schema seem less important based on our testing, but we still have them there as well and are passing them. We have a similar function call for event/schedules as you outlined @grav so I thought I’d share what we arrived at for a system instruction and it’s working decently well. Not 100% on every prompt but I’d give it 90th percentile.
You have access to a function through a API that provides the conference schedule, including meeting titles, dates, times, and locations. You MUST use this API when responding to questions about events. If you are listing meetings for users, always include the date, time, and location of each meeting, if available.
I was having the same issue and was able to make the model a bit more deterministic using at the end: # Given these instructions, call the tool my_function NOW
Also, because the cost is not so high, I make 5 API calls and the first that arrives without malformed response is executed and the workers are canceled.
Hi Grav, To resolve the inconsistencies with the Gemini API’s function calling. Here are some potential strategies to address the issues you’re experiencing:
Clarify Function Definitions: Make sure your function descriptions are clear and provide specific examples of inputs and expected outputs.
Reframe Requests: Explicitly frame your requests to emphasize the use of functions, such as, “Please use get_class_schedule(class_names).”
Simplify Conversation History: Keep the conversation history focused and concise to help the model concentrate on the current task.
Adjust Temperature: Slightly increase the temperature (e.g., to 0.1) to encourage more relevant responses while maintaining some consistency.
Log and Analyze Responses: Track instances where the model fails to call functions correctly and analyze them for patterns.
Engage with Community: Seek insights from developer forums and revisit the documentation for any updates or best practices.