Two Tool Calling Bugs I found in OpenAI Compatibility Beta

Gemini API
,
1

We’re looking for drop in replacements for OpenAI for a few reasons:

  • we can fall back to when their API is down
  • similar results at a lower pricepoint
  • better results at the same pricepoint

Dropping in the Gemini endpoint and running some tests, the following errors prevent us from going to production with Gemini:

  1. tool calls are returned under the wrong key toolCalls instead of tool_calls which breaks compatibility.

Example:

ChatCompletion(id=None, choices=[
  Choice(finish_reason=None, index=0, logprobs=None, message=
    ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, 
function_call=None, 
tool_calls=None, 
toolCalls=[{'function': {'arguments': '{"timezone":"America/New_York"}', 'name': 'get_datetime'}, 'id': '0', 'type': 'function'}]), finishReason='stop')], 
created=1735576664, model='gemini-1.5-pro-latest', object='chat.completion', service_tier=None, system_fingerprint=None, 
usage=CompletionUsage(completion_tokens=None, prompt_tokens=None, total_tokens=None, completion_tokens_details=None, prompt_tokens_details=None, completionTokens=9, promptTokens=100, totalTokens=109)
)

Here’s how the pydantic BaseModel is configured in the openAI python library:

class BaseModel(pydantic.BaseModel):
    if PYDANTIC_V2:
        model_config: ClassVar[ConfigDict] = ConfigDict(
            extra="allow", defer_build=coerce_boolean(os.environ.get("DEFER_PYDANTIC_BUILD", "true"))
        )

extra="allow" means that any keys that don’t match the expected schema are stored, thus why the toolCalls key is present, but this formatting issue prevents Gemini from being a true drop in replacement.

  1. Gemini API does not allow function definitions without any arguments:

consider the following:


def get_current_time() -> str:
    datetime.now().strftime("%Y-%m-%d %H:%M:%S")

tools = [
{
        "type": "function",
        "function": {
            "name": "get_current_time",
            "description": "Get current time in UTC",
            "parameters": {
                "type": "object",
                "properties": {
                },
                "required": [],
            },
        },
    },
]

if tools:
    params["tools"] = tools
    params["tool_choice"] = "auto"

response: ChatCompletion = await self.client.chat.completions.create(
    **params
)

This will produce the following error: openai.BadRequestError: Error code: 400 - [{'error': {'code': 400, 'message': 'Request contains an invalid argument.', 'status': 'INVALID_ARGUMENT'}}]

NOTE: I ran into the same issue while trying to implement using the genai library, so this might be a general Gemini backend issue rather than an OpenAI specific issue, however it does break compatibility with the OpenAI API.

Tested a bunch of different cases, and this was the only case that produced a 400 error.

2 Likes
2

Welcome to the forum. Hopefully the bug with parameterless tool functions will get fixed, it has been escalated to the developers according to this - Gemini doesn't accept parameterless functions - #13 by GUNAND_MAYANGLAMBAM

1 Like
3

wonderful, hopefully the other issue gets escalated and quickly resolved, seeing as it’s really just a camelCase vs. snake_case issue.

1 Like
5

Don’t know if it will help your specific situation, but I hit the “no parameters” problem as well, and just dropped parameters entirely. It’s optional in the schema. Eg:

  get schema(): ChatCompletionTool {
    return {
      type: "function" as const,
      function: {
        name: this.name,
        description: "Get information about the current AI model being used, including its capabilities and configuration.",
      },
    };
  }

This made sense to me in retrospect. The other definition is one parameter: an object with nothing in it, not zero parameters.

3 Likes