openai.BadRequestError: Error code: 400 when try to generate structured nested output

Gemini API
1

I’m having issues when calling Gemini via openai API.
I tried to generate a structured nested output, so I used pydantic basemodel.

The code snippet:

class SellingPoint(BaseModel):
    Content: str
    Keywords: str

class ProductIntroduction(BaseModel):
    Introduction: str
    Selling_point_1: SellingPoint
    Selling_point_2: SellingPoint
    Selling_point_3: SellingPoint
    Conclusion: str

response = client.beta.chat.completions.parse(
    model="gemini-2.0-flash",
    messages=[
        {
            "role": "system",
            "content": sys_prompt},        
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": prompt
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/jpeg;base64,{e_img1}",
                    }
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/jpeg;base64,{e_img2}",
                    }
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/jpeg;base64,{e_img3}",
                    }
                }
            ]
        }],
    response_format=ProductIntroduction
)

The error occured:

Traceback (most recent call last):
  File "/data/home/hedy/ai_marketing_video/gemini.py", line 51, in <module>
    response = client.beta.chat.completions.parse(
  File "/home/hedy/anaconda3/envs/market/lib/python3.10/site-packages/openai/resources/beta/chat/completions.py", line 161, in parse
    return self._post(
  File "/home/hedy/anaconda3/envs/market/lib/python3.10/site-packages/openai/_base_client.py", line 1290, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
  File "/home/hedy/anaconda3/envs/market/lib/python3.10/site-packages/openai/_base_client.py", line 967, in request
    return self._request(
  File "/home/hedy/anaconda3/envs/market/lib/python3.10/site-packages/openai/_base_client.py", line 1071, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - [{'error': {'code': 400, 'message': 'Request contains an invalid argument.', 'status': 'INVALID_ARGUMENT'}}]

However, if I changed the output format into non-nested (flat) like below, it could execute well.

class ProductIntroduction(BaseModel):
    Introduction: str
    Selling_point_1: str
    Selling_point_2: str
    Selling_point_3: str
    Conclusion: str

Other information:
python version: 3.10.16
openai version: 1.64.0

Has anyone run into the error?

2

1000008911.jpg720×1600 129 KB

Maybe e-img3 error no items, not sure I’m right but u can try.

Best Regards

Lan

4

Hi Hedy,

I hope you are doing well,

I’ve encountered the same issue while using nested pydantic model.
It works fine for me too with a simple model.

Were you able to find a solution or an answer to your problem ?
It seems to me that the issue is from google side and that gemini is not able to support complex pydantic class.

Best regards,

Mathieu

5

Hi,

looking at the error response

it seems that the OpenAI code base generates a JSON Schema that is not (yet) accepted by the Gemini API. Gemini’s OpenAI compatibility is still marked as Experimental and its response_format is based on a select subset of the OpenAPI 3.03 specification which does not handle all attributes that OpenAI might generate.

References:

The Schema represents a select subset of the OpenAPI 3.0 Schema object.

I ran into a similar problem writing the SDK for .NET where record types are specified with the readOnly attribute which the Gemini API doesn’t accept and rejects as HTTP 400 Bad Request.

Cheers.

PS: Tagging @GUNAND_MAYANGLAMBAM and @Vishal

1 Like