Bug using required boolean fields with Gemini 2.0-models

Gemini API
1

I’ve found a reproduceable bug in Gemini 2.0 Flash.

System instructions:

Answer with JSON:
  - 'keywords': 1-3 keywords of the message
  - 'answered': True if the message has been fully answered by a support agent

JSON Schema:

{
  "type": "object",
  "properties": {
    "keywords": {
      "type": "string"
    },
    "answered": {
      "type": "boolean"
    }
  },
  "required": [
    "keywords",
    "answered"
  ]
}

Message:

User: How can I reset the password?

Support agent: Go to the reset password page and enter your email address.

User: Thanks, but where can I find the reset password page?

Here “answered” is true (expected was false). Changing the type of the field to String returns the expected “false”.

I’ve seen the same problem in larger production prompts as well. Gemini 1.5-Flash doesn’t have the same problem.

3

Hi @aleksabl,

Welcome to the forum! I attempted to reproduce the issue you described, but it appears to be working correctly on my end.Here is the gist: Google Colab. Could you please take a look and let me know if I’ve missed any steps in your process?

Cheers!