Gemini 2.0 Flash fails to generate a structured output

Gemini API
, , ,
1

I’m using the Gemini 2.0 Flash model (gemini-2.0-flash-001) through Vertex AI, passing a responseSchema with propertyOrdering. There are a few cases where the model fails to generate a valid structured output. I’m generating responses in portuguese, so we have a few special characters, such as ç, á ,ã ,õ and so on. It seems like the error always happens when these characters should be present in the output. When the error happens, I get a bunch of \n characters. Some examples:

{
  "justificativa": "O cliente deseja saber como transferir um produto da situa\nn\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n [hundreds of other \n]

In this instance the model should have produced the word “situação”

{
  "justificativa": "O cliente está com dificuldades para dar baixa em uma negativa\nnativa\n\n\n\n\n\n\n\n\n\n\n\n\n\n [hundreds of other \n]

In this instance the model should have produced the word “negativação”

There was a single instance where I’ve seen the model “recover” from the infinite \n loop:

{
  "justificativa": "O usu\nrio precisa de informa\n\n\n\n\n\n\n\n\n\n [hundreds of other \n] \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nI'm sorry, I cannot fulfill this request. I am unable to provide a response that adheres to the specified JSON schema, as the user's request is open-ended and requires a more nuanced understanding of the situation to determine the appropriate department. Providing a JSON response without this understanding would be inaccurate and potentially unhelpful.

In this instance the model should have produced the words “usuário” and “informações”

It doesn’t happen often, but I’d like to really trust that the output is a valid JSON string.

3

Hi @Daniel_Konflanz,

Sorry for the delay in response. I wasn’t able to replicate this problem on my end using gemini-2.0-flash. Can you confirm if you’re still encountering it?

Thank you!!

5

Hello! I’ve mostly migrated to Gemini 2.5 now, but yes, I’m still facing the issue. I was able to replicate it in a minimal code snippet, but I’m not sure I’m allowed to share the prompt publicly. I’ll DM the snippet to you. Btw, I’m facing another issue in Gemini 2.5, also related to JSON output, where I’m getting an exclamation point at the beginning of the response. Do you want me to create a new thread for that issue?

1 Like
7

The solution was a mixture of two things.

I realized tht the problem migth be that the max token output parameter is not properly accpeted by the IA.

So I increased it and change the output from plain/test to application/json.

Hope this works, at least in 2.0-flash works :slight_smile: