Gemini 1.5-flash Function Calling Not Working (Works on Gemini 1.5-pro)

Gemini API
,
1

I’m encountering an issue with function calling using the Gemini 1.5-flash model. Function calls are working correctly with the Gemini 1.5-pro model, but consistently fail with the flash model despite identical configurations.

Both models are used with the same client setup:

client = genai.GenerativeModel(
  model_name= gemini.model_name,
  generation_config= gemini.generation_config,
  system_instruction=gemini.system_instructions,
  tools=gemini.tools,
  tool_config={'function_calling_config':'AUTO'},
)

Where gemini.generation_config, gemini.system_instructions, and gemini.tools are identical for both the flash and pro model invocations. The chat proceeds normally, and the model generates responses appropriately, but when a function call is expected, the flash model fails to invoke any functions while the pro model works as intended.

We’re trying to utilize the flash model for cost optimization in our project, but this function calling failure is blocking our progress.

Could you please advise on potential causes for this discrepancy? Are there known limitations or configuration differences specific to function calling with the Gemini 1.5-flash model that aren’t present in the pro model? Any troubleshooting steps or suggestions would be greatly appreciated.

1 Like
3

The very same experience here, the moment I swap the model to Pro it works like a charm.

5

Hi @KonradPs, @edin, welcome to the forum…

If possible, could you provide me with a sample code so I can reproduce the issue on my end?

Thanks

6

I created this experiment:

from typing import Any

from vertexai.generative_models import GenerativeModel, GenerationConfig, FunctionDeclaration, Tool, Content, Part, \
    GenerationResponse

model_instructions = """
    You're a general helper bot. You can help with a variety of tasks.
"""
model = GenerativeModel(
    model_name="gemini-1.5-pro",
    generation_config=GenerationConfig(temperature=0.5),
    system_instruction=model_instructions,
)

time_tool = FunctionDeclaration(
    name="time_tool",
    description="This tool returns current time.",
    parameters={
        "type": "object",
        "properties": {},
        "required": []
    }
)

def handle_time_tool() -> str:
    return "The current time is 23:59 PM"

tools = Tool(function_declarations=[time_tool])

user_prompt_content = Content(
    role="user",
    parts=[Part.from_text("What is the current time?")]
)

response = model.generate_content(
    [user_prompt_content],
    tools=[tools]
)


def extract_function_calls(response: GenerationResponse) -> list[dict]:
    function_calls: list[dict] = []
    if response.candidates[0].function_calls:
        for function_call in response.candidates[0].function_calls:
            function_call_dict: dict[str, dict[str, Any]] = {
                function_call.name: {
                    "args": function_call.args
                }
            }
            function_calls.append(function_call_dict)
    return function_calls


function_calls = extract_function_calls(response)

while len(function_calls) > 0:
    parts = []
    for function_call in function_calls:
        func_name, func_args = list(function_call.items())[0]

        if func_name == "time_tool":
            resp = handle_time_tool()
            parts.append(Part.from_function_response(
                name="time_tool",
                response={"content": resp},
            ))

    updated_history = [user_prompt_content,response.candidates[0].content, Content(parts=parts)]
    response = model.generate_content(
        updated_history,
        tools=[tools],
    )
    function_calls = extract_function_calls(response)

print(response.candidates[0].content.parts[0].text)

When using gemini-1.5-pro, the return text is: The current time is 23:59 PM.
But when I switch to gemini-1.5-flash-002 the return text is: The available tools don’t provide the current time. I need access to a time-providing API or library to answer that question.
When I tried gemini-1.5-flash-001 the returned text was correct: The current time is 23:59 PM.

8

Hey @KonradPs, thanks for the sample code. I also tried gemini-1.5-flash, and it gives the correct response. It seems like there is an issue with the latest gemini-1.5-flash-002 model. I have escalated this issue with the team.

Thanks

2 Likes
9

I have the same issue for couple months. i think google resolve this but the issue still exist. the flash model not working exactly. i send exact text just some times the model call function and most of times don’t call the function. but the pro model is working.