MALFORMED_FUNCTION_CALL finish reason happens too frequently with vertex AI

Gemini API
,
1

I often get the MALFORMED_FUNCTION_CALL finish reason. This is issue is extremely frequent with Vertex AI and less frequent with Google AI developer API. To repro, use this

import google.genai as genai
from dotenv import load_dotenv
import os

load_dotenv()

client = genai.Client(
    vertexai=True, project=os.getenv("GOOGLE_CLOUD_PROJECT"), location="us-central1"
)

tool_def = {
    "name": "think",
    "description": "Call this tool to think about a given thought.",
    "parameters": {
        "properties": {
            "thought": {
                "description": "The thought to think about.",
                "type": "string",
            }
        },
        "required": ["thought"],
        "type": "object",
    },
}

response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents=[{"role": "user", "parts": [{"text": "Hi, how's the weather in Tokyo?"}]}],
    config={
        "tools": [{"function_declarations": [tool_def]}],
        "system_instruction": "You have the tool get_weather available. You ABSOLUTELY MUST call the tool get_weather to answer the user's question.",
        "thinking_config": {"thinking_budget": 24576, "include_thoughts": True},
    },
)
response.model_dump()

6 out of 7 times, the response has the MALFORMED_FUNCTION_CALL finish reason. Now, I know that the system prompt "You have the tool get_weather available. You ABSOLUTELY MUST call the tool get_weather to answer the user's question." is very perverse because it is intentionally trying to trick the model; but I did this to make the issue more reproducible. In my actual production environment, the system prompt looks more like this "You will get the tool get_weather at some point in the future. You ABSOLUTELY MUST call the tool get_weather to answer the user's weather question.".

The interesting phenomenon is that this issue is much less frequent but still present when using the Google AI Developer API. Run the above repro code with the Google AI Developer API and you will get this issue like 1 out of 10 times.

2

why is malformed function call even handled at the server level? I understand that malformed function calls can happen for any model, but literally any other model provider (OpenAI, Anthropic, etc.) still returns the malformed function call as a regular response. I prefer this because I want to handle the malformed function call at my app layer, which allows me to do things like telling the model "You called tool_x incorrectly in the following way ... please do the following ... to fix it"

3

to me, the ideal fix is to disable server-side MALFORMED_FUNCTION_CALL or expose a flag to disable it. Therefore, malformed function calls are returned as regular responses and developers become responsible for handling them.