Function calling issues. with type object and array

Gemini API
1

This function appears to work fine. In that it does parse and does not throw an error.

search = genai.protos.Tool(
function_declarations=[
genai.protos.FunctionDeclaration(
name=‘search’,
description=“Searches for a number within a sorted array”,
parameters=genai.protos.Schema(
type=genai.protos.Type.OBJECT,
properties={
‘arr’: genai.protos.Schema(type=genai.protos.Type.BOOLEAN),
‘x’: genai.protos.Schema(type=genai.protos.Type.NUMBER)
},
)
)
])

However if i set the type to Object

'arr': genai.protos.Schema(type=genai.protos.Type.OBJECT),

I get an error

google.api_core.exceptions.InvalidArgument: 400 * GenerateContentRequest.tools[0].function_declarations[0].parameters.properties[arr].properties: should be non-empty for OBJECT type

If i set it to Array

'arr': genai.protos.Schema(type=genai.protos.Type.ARRAY),

I again get an error

google.api_core.exceptions.InvalidArgument: 400 * GenerateContentRequest.tools[0].function_declarations[0].parameters.properties[arr].items: missing field.

The main issue being here is that arr has not been set to required meaning that it should not be a missing field.

Beyond that I have tried serval ways of getting it to detect both Object and Array and have been unable

It does not catch this for example.

response = model.generate_content((f'This is an array [12,12,411,14]'))

It still returns missing.

Minimal Reproducible Example:

from dotenv import load_dotenv
import google.generativeai as genai
import os

load_dotenv()
genai.configure(api_key=os.environ['API_KEY'])
MODEL_NAME_LATEST = os.environ['MODEL_NAME_LATEST']


def multiply(a: float, b: float):
    """returns a * b."""
    return a * b


search = genai.protos.Tool(
    function_declarations=[
        genai.protos.FunctionDeclaration(
            name='search',
            description="Searches for a number within a sorted array",
            parameters=genai.protos.Schema(
                type=genai.protos.Type.OBJECT,
                properties={
                    'list': genai.protos.Schema(type=genai.protos.Type.ARRAY),
                     #'obj': genai.protos.Schema(type=genai.protos.Type.OBJECT),
                    'number': genai.protos.Schema(type=genai.protos.Type.NUMBER),
                    'find': genai.protos.Schema(type=genai.protos.Type.BOOLEAN)
                },
            )
        )
    ])

model = genai.GenerativeModel(model_name=MODEL_NAME_LATEST,
                              tools=[search])

response = model.generate_content((f'Find: true, number: 12 : List is [12,12,411,14]'))

fc = response.candidates[0].content.parts[0].function_call
print(fc.name)
if fc.name == 'search':
    if 'list' in fc.args:
        print("list is: ", fc.args['list'])
    if 'obj' in fc.args:
        print("Obj is: ", fc.args['obj'])
    if 'find' in fc.args:
        print("find is: ", fc.args['find'])
    if 'number' in fc.args:
        print("number is: ", fc.args['number'])
else:
    print(fc.name)
2

This error is because you’ve defined an object, but haven’t defined the attributes that the object has. Defining a schema with an empty object doesn’t make sense.

This is a little different. Arrays in the schema need to be able to store a particular type. Think of it as “an array of numbers”. You can’t just have “an array”. So this error is saying that you haven’t defined the type of the members of the array.

3

More specifically, the definitions that you have in the FunctionDeclaration for the OBJECT properties are incomplete. At least for the “list” and “obj” properites you were trying to define.

My python isn’t great, and I haven’t tested this, but based on what you have there and on the REST schema definition, I think those two lines should look something more like this:

                    'list': genai.protos.Schema(
                        type=genai.protos.Type.ARRAY,
                        items=genai.protos.SCHEMA(
                            type=genai.protos.Type.NUMBER
                        )
                    ),
                    'obj': genai.protos.Schema(
                        type=genai.protos.Type.OBJECT,
                        properties={
                            'numberValue': genai.protos.Schema(type=genai.protos.Type.NUMBER),
                            'stringValue': genai.protos.Schema(type=genai.protos.Type.STRING),
                            # Please don't make me do a property that is also an object
                        }
                    ),

With this, we’re defining that the “list” property is an array that contains numbers, and the “obj” property is an object that has two properties, one named “numberValue” which contains a number and the other named “stringValue” that contains a string.

4

You are my hero @afirstenberg I was banging my head against this for two hours this morning

search = genai.protos.Tool(
    function_declarations=[
        genai.protos.FunctionDeclaration(
            name='search',
            description="Searches for a number within a sorted array",
            parameters=genai.protos.Schema(
                type=genai.protos.Type.OBJECT,
                properties={
                    'list': genai.protos.Schema(type=genai.protos.Type.ARRAY,
                                                items=genai.protos.Schema(type=genai.protos.Type.INTEGER),                                                ),
                     'obj': genai.protos.Schema(type=genai.protos.Type.OBJECT,
                                                nullable=True,
                                                properties={"a": {"type_": genai.protos.Type.INTEGER}}),
                    'number': genai.protos.Schema(type=genai.protos.Type.NUMBER),
                    'find': genai.protos.Schema(type=genai.protos.Type.BOOLEAN)
                },
            )
        )
    ])
1 Like