Invalid JSON Output - Generation Stops at Line Break

Gemini API
1

Description:
I am experiencing an issue where the model’s JSON output is invalid and always stops at a line break. It seems to be occurring consistently, regardless of the prompt.

Steps to Reproduce:

  1. Use this prompt:
Enhance the content of the provided JSON with an interview question and example answer by:
- Elaborating answer if it is incomplete or unclear to provide clarity and completeness.
- Ensuring that question is clear and formulated in an appropriate way for an interview.
- Refining the overall structure of the answer to ensure the text is clear, logical, and well-organized.
- Amplifying examples or use cases to ensure they clearly demonstrate their intended purpose.
- Correcting any grammatical, spelling, or punctuation errors.
- Removing any redundant or repetitive information to make the text more concise.
- Adding transitional phrases where necessary to improve the flow of the content.
- Using clear formatting that makes the answer easy to read.
- Ensuring the technical terms and jargon are used appropriately and accurately.
- Adjusting the tone of the content to a level of 5 on a scale from informal jargon (0) to high corporate formal language (10).

Constraints:
- Use language of the input, regardless of the language used.

Use this JSON schema for output:
{ 
  "type": "object",
  "properties": {
    "question": { "type": "string" },
    "answer": { "type": "string" },
  }
}

Input JSON:
{"question": "What keyword is used to declare a catch block in exception handling?", "answer": "The keyword for declaring a catch block in exception handling in most programming languages, such as Java, C++, C#, and Python, is \"catch\". The catch block follows the try block and is used to catch exceptions that may occur during the execution of the code within the try block. Inside the catch block, you can handle the exception, such as displaying an error message, performing some recovery, or simply ignoring it. For example, in Java, the code might look like this:\n\ntry {\n // Code that may throw an exception\n} catch (Exception e) {\n // Handle the exception\n}"}
  1. Run 1.5 Flash with the specified input. Json output is ON.
  2. Observe the JSON output. The model’s response consistently stops at a line break, rendering the JSON invalid. Happens in 90% of cases. Works with some json inputs, with others works fine.

Expected Behavior:
The model should generate a valid JSON response.

Actual Behavior:
The JSON output stops at a line break, making it invalid.

Additional Information:

  • I have tried restarting the model and clearing the cache, but the issue persists.
  • I am using the Gemini 1.5 Flash model. Pro 1.5 also fails.
  • Smaller prompts usually works fine.
  • No JSON output makes it work fine.
  • Problem presists in studio and via API.

image
image3838×1614 474 KB

1 Like
2

Welcome to the forum.

I got the same result, in AI Studio the response is complete when the Output in JSON selector is off, and is truncated when the selector is on.

I think the problem are the three backticks. In the example you show, and in three generation examples I tried out, there are newlines in the response that did not interrupt the output. In all examples, what would have been generated next is a code snippet, and the model generates the three-backtick sequence to fence it off. That’s what breaks the JSON, in my opinion.

If you agree, I recommend editing the topic heading to reflect the actual culprit.

3

This modified prompt succeeds in JSON mode:


Enhance the content of the provided JSON with an interview question and example answer by:
- Elaborating answer if it is incomplete or unclear to provide clarity and completeness.
- Ensuring that question is clear and formulated in an appropriate way for an interview.
- Refining the overall structure of the answer to ensure the text is clear, logical, and well-organized.
- Amplifying examples or use cases to ensure they clearly demonstrate their intended purpose.
- Correcting any grammatical, spelling, or punctuation errors.
- Removing any redundant or repetitive information to make the text more concise.
- Adding transitional phrases where necessary to improve the flow of the content.
- Using clear formatting that makes the answer easy to read.
- Ensuring the technical terms and jargon are used appropriately and accurately.
- Adjusting the tone of the content to a level of 5 on a scale from informal jargon (0) to high corporate formal language (10).

Constraints:
- Use language of the input, regardless of the language used.

Use this JSON schema for output:
{ 
  "type": "object",
  "properties": {
    "question": { "type": "string" },
    "answer": { "type": "string" },
    "code_example":  { "type": "string" }
  }
}

Input JSON:
{"question": "What keyword is used to declare a catch block in exception handling?", "answer": "The keyword for declaring a catch block in exception handling in most programming languages, such as Java, C++, C#, and Python, is \"catch\". The catch block follows the try block and is used to catch exceptions that may occur during the execution of the code within the try block. Inside the catch block, you can handle the exception, such as displaying an error message, performing some recovery, or simply ignoring it. For example, in Java, the code might look like this:",
"code_example": "try \{\n // Code that may throw an exception\n\} catch (Exception e) \{\n // Handle the exception\n\}"}

The code_example in the schema separates the part that would have required fencing. The model response is proper JSON.

Hope that helps!