Function not being called when using NodeJS VertexAI SDK

Gemini API
,
1

In the past day or so, our code that asks Gemini (2.x, doesn’t seem to matter which model) to call a function with data is now producing inline text with and tags surrounding python code.

We upload data in JSON format for the model to work with, and this issue appears related to the size of that data set. If the data set is small, perhaps 10KB, the model calls our function as requested in the system instructions. If the data set is larger, perhaps over 1MB, the model instead includes the data that should go to the function inline.

The snippet of inline code the model produces always takes this form:

<ctrl97>tool_code
print(default_api.store_results(values=[{
  "rules": [
    {
      "id": "metric_1",
      "name": "Channel",
      "tab": "Channel",
      "value": -129.7696269166154,
      "change_type": "fromCurrent",
      "interval_type": "absolute"
    }
  ],
  "start_date": "2026-01",
  "end_date": "2026-12",
  "changes_name": "5% Uplift (Channel Spend)",
  "changes_summary_text": "Channel update"
}]))<ctrl98>

We’ve seen embedded Python and JSON in responses before, but it’s been due to badly-worded instructions. This suddenly started happening in all our queries in the past 24 hours and is always included instead of calling our function, assuming our included data is large enough.

Any ideas? Has anyone else seen this behavior?

1 Like
2

It looks like the issue is due to the model’s token limit being exceeded when handling large inputs. When the input size grows, the model may attempt to include the entire dataset within the response, leading to the observed inline code.

Try reducing input size or use streaming.

3

This has started happening for us as well in the past couple days using the google.golang.org/genai Go SDK for almost all of our requests. The output is just like you described. When the model wants to make tool calls it will now output text like:

First, I need to understand the available APIs.
<ctrl97>tool_code
print(default_api.getPackageDocs(packageName="[snip]"))
print(default_api.getPackageDocs(packageName="[snip]"))<ctrl98>

Previously these would have been actual function call requests which we could access via resp.Candidates.Parts[...].FunctionCall. Now there is only a Text part, no parts with FunctionCall at all.

It does seem like reducing the input size fixes the issue but that limits the usefulness a lot.

It happens both when streaming and not streaming the output. I’m not sure if that is what you’re referring to or not?

1 Like