500 error on mime type 'application/json'

Gemini API
1 
message = model.generate_content(
            prompt,
            generation_config=genai.types.GenerationConfig(
                response_mime_type="application/json",
                response_schema=schema,
                temperature=1.0,
                max_output_tokens=8192,
                top_p=0.95,
                top_k=64
            )
        )

With this code I get a 500 Internal error; if i use mime type text/plain or if I remove response_mime_type I don’t get the error.

2

Welcome to the forums!

What model are you trying to use JSON Mode with?

3

Try this.

generation_config_json: str = {
        'temperature': 0.9,
        'top_p': 1,
        'top_k': 40,
        'max_output_tokens': 2048,
        'stop_sequences': [],
        "response_mime_type": "application/json",
    }

    generation_config: str = {
        'temperature': 0.9,
        'top_p': 1,
        'top_k': 40,
        'max_output_tokens': 2048,
        'stop_sequences': [],
    }

    safety_settings: list[str] = [{"category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_NONE"},
                                  {"category": "HARM_CATEGORY_HATE_SPEECH", "threshold": "BLOCK_NONE"},
                                  {"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "threshold": "BLOCK_NONE"},
                                  {"category": "HARM_CATEGORY_DANGEROUS_CONTENT", "threshold": "BLOCK_NONE"}]


model = genai.GenerativeModel(model_name=os.getenv("TEXT_MODEL_NAME_LATEST"),
                                           system_instruction=GeminiService.read_prompt(prompt_path),
                                           generation_config=GeminiService.generation_config_json if as_json else GeminiService.generation_config,
                                           safety_settings=self.safety_settings)
4

I just started getting this error too (my code has been working fine for weeks). When I remove responseMimeType: "application/json", it works.

Error:

GoogleGenerativeAIFetchError: [GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent: [500 Internal Server Error] An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting

EDIT: I tried it with both pro and flash. Both don’t work.

1 Like
5

Hi @Daniele_Gargini,

I was able to call the api without any error can you take a look below and also attached a working gist of the same:

prompt = """Write a short story and list its characters and story using this JSON schema:

Character = {"name": str}
Story = {"story": str, "characters": list[Character]}
Return: list[Story]"""

response = model.generate_content(
    prompt,
    generation_config=genai.types.GenerationConfig(
        # Only one candidate for now.
        candidate_count=1,
        max_output_tokens=8192,
        temperature=1.0,
        response_mime_type="application/json",
    ),
)

print(response)

Also while using Flash

While gemini-1.5-flash models only accept a text description of the JSON you want back, gemini-1.5-pro models support “controlled generation” (aka “constrained decoding”). This allows you to pass a schema object (or a python type equivalent) and the output will strictly follow that schema.

Thanks !

6

Hi everyone, I was using ‘gemini-1.5-pro’.
Now it seems to be working again, thank you all.

7

Hi, what’s the response_schema parameter for and when has it been published?

8

It showed up as a member in GenerationConfig  |  Google AI for Developers  |  Google for Developers to support this update Gerar saída JSON com a API Gemini  |  Gemini API  |  Google for Developers about a month ago, maybe a bit more. The feature works with Gemini 1.5 Pro; Gemini 1.5 Flash only understands the textual json description in the prompt (and is therefore more likely to make mistakes, although in my experience Flash gets it right too).

2 Likes
9

Ciao @Daniele_Gargini, Hi all,

If you have something that’s working and it suddenly stops working and returns a 5XX error, then it’s a model problem. It could have gone off-line or be too busy, as in the case that in the same minute, the same model gives error 500 if the JSON format is requested and instead responds correctly if the format is text.

The advice is to take a break and try again after a few minutes.

If you’re using the ‘v1beta’ API, well, that’s it, it’s beta :slight_smile:

More here about 5XX error codes: Troubleshooting guide  |  Gemini API  |  Google AI for Developers