Tool calling with OpenAI API not working

Gemini API
,
1

I’m trying to use the OpenAI Node.js SDK to use function calling on Gemini using this example: Compatibilité OpenAI  |  Gemini API  |  Google AI for Developers

But I get a 400 response when I provide the function result.

The example in the documentation doesn’t cover the function response.

Here is the messages object I provide in the call:

[
  {
    "role": "system",
    "content": "My system prompt"
  },
  {
    "role": "user",
    "content": "number of customers using xxx"
  },
  {
    "role": "assistant",
    "tool_calls": [
      {
        "function": {
          "arguments": "{\"query\":\"\\n[\\n    {\\n        \\\"$match\\\": {\\n            \\\"kind\\\": \\\"xxx\\\"\\n        }\\n    },\\n    {\\n        \\\"$group\\\": {\\n            \\\"_id\\\": \\\"$customer\\\",\\n            \\\"count\\\": {\\n                \\\"$sum\\\": 1\\n            }\\n        }\\n    },\\n    {\\n        \\\"$count\\\": \\\"total\\\"\\n    }\\n]\\n\"}",
          "name": "execute_mongo_aggregate_query"
        },
        "id": "0",
        "type": "function"
      }
    ]
  },
  {
    "role": "tool",
    "content": "{\"data\":[{\"total\":24}]}",
    "tool_call_id": "0"
  }
]

The same is working with the OpenAI API

2

The API does not support the “legacy” functions parameter.

It only will accept tools.

Also note that the Google API compatibility isn’t correctly returning the index field in tool_calls, employed for OpenAI models supporting parallel tool calls, which has been there since the introduction of “tools”, which also match an ID between tool call and tool return, for ordering context. I haven’t yet unraveled further incompatibility in matching conversational history.

You might omit sending back a tool call ID and see how that performs.

1 Like
4

I’m still not able to get this to worrk:

ERROR:root:❌ HTTP error occurred: 400 Client Error: Bad Request for url: https://generativelanguage.googleapis.com/v1beta/openai/chat/completions
ERROR:bot.llm.base_llm:Data sent: {
    "messages": [
        {
            "role": "system",
            "content": "Just do as told"
        },
        {
            "role": "user",
            "content": "Hi"
        }
    ],
    "model": "gemini-2.0-flash"
}
ERROR:bot.llm.base_llm:Response: [
    {
        "error": {
            "code": 400,
            "message": "Request contains an invalid argument.",
            "status": "INVALID_ARGUMENT"
        }
    }
]
5

I have the same issue here.

There is no documentation of what a valid JSON tool response might look like

If you simply send without the tool execution response, it works


   {
         "role": "user",
         "content": "remember that I am 52 years old "
      },
      {
         "role": "assistant",
         "tool_calls": [
            {
               "function": {
                  "arguments": "{\"argument_1\":\"[{\\\"key_name\\\": \\\"age\\\", \\\"key_value\\\":\\\"52\\\", \\\"key_description\\\": \\\"The age of the user\\\", \\\"key_namespace\\\": \\\"personal_data\\\"}]\",\"argument_2\":\"False\"}",
                  "name": "set_in_user_memory"
               },
               "id": "",
               "type": "function"
            }
         ]
      },

but this doesn’t

      {
         "role": "user",
         "content": "remember that I am 52 years old "
      },
      {
         "role": "assistant",
         "tool_calls": [
            {
               "function": {
                  "arguments": "{\"argument_1\":\"[{\\\"key_name\\\": \\\"age\\\", \\\"key_value\\\":\\\"52\\\", \\\"key_description\\\": \\\"The age of the user\\\", \\\"key_namespace\\\": \\\"personal_data\\\"}]\",\"argument_2\":\"False\"}",
                  "name": "set_in_user_memory"
               },
               "id": "",
               "type": "function"
            }
         ]
      },
      {
         "tool_call_id": "",
         "role": "tool",
         "name": "set_in_user_memory",
         "content": "The parameters were successfully saved or updated in the user memory."
      }

The tool_call from Gemini through the openAI doesn’t have a valid tool ID (and index is missing)
to make it work, you need to set the value of the id with the tool function name for example, and make sure it matches your tool_call_id value

but this does

      {
         "role": "user",
         "content": "remember that I am 52 years old "
      },
      {
         "role": "assistant",
         "tool_calls": [
            {
               "function": {
                  "arguments": "{\"argument_1\":\"[{\\\"key_name\\\": \\\"age\\\", \\\"key_value\\\":\\\"52\\\", \\\"key_description\\\": \\\"The age of the user\\\", \\\"key_namespace\\\": \\\"personal_data\\\"}]\",\"argument_2\":\"False\"}",
                  "name": "set_in_user_memory"
               },
               "id": "**set_in_user_memory**",
               "type": "function"
            }
         ]
      },
      {
         "tool_call_id": "**set_in_user_memory**",
         "role": "tool",
         "name": "**set_in_user_memory**",
         "content": "The parameters were successfully saved or updated in the user memory."
      }

Voila!

2 Likes