Structured output Optional fields don't work in OpenAI SDK compatibility mode

Gemini API
, ,
1

If I submit a model like the below as response_format to OpenAI structured output it won’t complain. The same isn’t the case with Gemini’s OpenAI endpoint https://generativelanguage.googleapis.com/v1beta/openai/ , which returns [{'error': {'code': 400, 'message': 'Request contains an invalid argument.', 'status': 'INVALID_ARGUMENT'}}]

Does someone know how to make this work without hacky workarounds or alternatives?

class MyModel(BaseModel):
    value: str | None
3

Hi @Hayk_Harutyunyan, I have tried to generate the response from the Gemini models using open_api library and was able to pass that response to the defined pydantic class.

from openai import OpenAI

client = OpenAI(
    api_key=api,
    base_url="https://generativelanguage.googleapis.com/v1beta/openai/"
)

response = client.chat.completions.create(
    model="gemini-2.0-flash",
    messages=[
        {"role": "system", 
         "content": "You are a helpful assistant."},
        
        {"role": "user",
         "content": "Hi, How are you?"
        }
    ]
)
class MyModel(BaseModel):
    value: str | None

MyModel(value=response.choices[0].message.content).value
#output: Hi there! I am doing well, thank you for asking. How can I help you today?

I did not face any error. Could you please confirm if I am missing anything?

Thank You.

5

Thanks for reaching back. The ‘canonical’ way would be to use the .parse method to do the parsing automatically and avoid
MyModel(value=response.choices[0].message.content).value

Here is an example that would create that error

import os
from openai import OpenAI
from pydantic import BaseModel



class MyModel(BaseModel):
    value: str | None



client = OpenAI(
    api_key=os.environ["GEMINI_SECRET"],
    base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
)


messages = [
    {
        "role": "user",
        "content": "What is the capital of Armenia?",
    }
]

response = client.beta.chat.completions.parse(
        model="gemini-2.0-flash",
        messages=messages,
        response_format=MyModel,
)

parsed = response.choices[0].message.parsed

Strangely enough I just tried gemini-2.5-flash-preview-04-17 for the model and it worked without an issue.