There is an issue with Gemini: it sometimes returns the correct response and other times gives a 500 error

Gemini API
, ,
1

I am using Python and have written the following code based on the official documentation.
I tried to submit feedback, but for some reason, I am unable to do so.
Here is the Python code:

import os 
from dotenv import load_dotenv
import typing_extensions as typing
import google.generativeai as genai



load_dotenv()


API_KEY = os.getenv("API_KEY")


genai.configure(api_key=API_KEY)



class Recipe(typing.TypedDict):
    recipe_name: str
    ingredients: list[str]

model = genai.GenerativeModel("gemini-1.5-pro-latest")
result = model.generate_content(
    "List a few popular cookie recipes.",
    generation_config=genai.GenerationConfig(
        response_mime_type="application/json", response_schema=list[Recipe]
    ),
)
print(result)

I received a response the first two times, but then I encountered an error. Here are the complete logs:

response:
GenerateContentResponse(
    done=True,
    iterator=None,
    result=protos.GenerateContentResponse({
      "candidates": [
        {
          "content": {
            "parts": [
              {
                "text": "[{\"ingredients\": [\"flour\", \"sugar\", \"butter\", \"eggs\", \"vanilla extract\"], \"recipe_name\": \"Chocolate Chip Cookies\"}, {\"ingredients\": [\"flour\", \"sugar\", \"butter\", \"peanut butter\", \"eggs\"], \"recipe_name\": \"Peanut Butter Cookies\"}, {\"ingredients\": [\"flour\", \"sugar\", \"butter\", \"eggs\", \"oatmeal\", \"raisins\"], \"recipe_name\": \"Oatmeal Raisin Cookies\"}, {\"ingredients\": [\"flour\", \"sugar\", \"butter\", \"cocoa powder\", \"eggs\"], \"recipe_name\": \"Double Chocolate Cookies\"}, {\"ingredients\": [\"flour\", \"sugar\", \"butter\", \"eggs\", \"lemon zest\"], \"recipe_name\": \"Lemon Sugar Cookies\"}] "
              }
            ],
            "role": "model"
          },
          "finish_reason": "STOP",
          "index": 0,
          "safety_ratings": [
            {
              "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
              "probability": "NEGLIGIBLE"
            },
            {
              "category": "HARM_CATEGORY_HATE_SPEECH",
              "probability": "NEGLIGIBLE"
            },
            {
              "category": "HARM_CATEGORY_HARASSMENT",
              "probability": "NEGLIGIBLE"
            },
            {
              "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
              "probability": "NEGLIGIBLE"
            }
          ]
        }
      ],
      "usage_metadata": {
        "prompt_token_count": 8,
        "candidates_token_count": 154,
        "total_token_count": 162
      }
    }),
)
(gemini) (base) ishaquenizamani@ishaquenizamani-ThinkPad-T460:~/work/opensourse/GoAndPyMasters/simpepython$ python main.py 
Traceback (most recent call last):
  File "/home/ishaquenizamani/work/opensourse/GoAndPyMasters/simpepython/main.py", line 23, in <module>
    result = model.generate_content(
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ishaquenizamani/anaconda3/envs/gemini/lib/python3.11/site-packages/google/generativeai/generative_models.py", line 331, in generate_content
    response = self._client.generate_content(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ishaquenizamani/anaconda3/envs/gemini/lib/python3.11/site-packages/google/ai/generativelanguage_v1beta/services/generative_service/client.py", line 830, in generate_content
    response = rpc(
               ^^^^
  File "/home/ishaquenizamani/anaconda3/envs/gemini/lib/python3.11/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__
    return wrapped_func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ishaquenizamani/anaconda3/envs/gemini/lib/python3.11/site-packages/google/api_core/retry/retry_unary.py", line 293, in retry_wrapped_func
    return retry_target(
           ^^^^^^^^^^^^^
  File "/home/ishaquenizamani/anaconda3/envs/gemini/lib/python3.11/site-packages/google/api_core/retry/retry_unary.py", line 153, in retry_target
    _retry_error_helper(
  File "/home/ishaquenizamani/anaconda3/envs/gemini/lib/python3.11/site-packages/google/api_core/retry/retry_base.py", line 212, in _retry_error_helper
    raise final_exc from source_exc
  File "/home/ishaquenizamani/anaconda3/envs/gemini/lib/python3.11/site-packages/google/api_core/retry/retry_unary.py", line 144, in retry_target
    result = target()
             ^^^^^^^^
  File "/home/ishaquenizamani/anaconda3/envs/gemini/lib/python3.11/site-packages/google/api_core/timeout.py", line 120, in func_with_timeout
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ishaquenizamani/anaconda3/envs/gemini/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InternalServerError: 500 An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting
3

Are you experiencing the same issue with gemini-1.5-flash as well?

4

I am using
gemini-1.5-pro-lates
but yeah same 500 error

5

While 500 errors aren’t uncommon (best practice is to retry), there are two things to look out for:

  • If you’re seeing them frequently, or only, for the same or similar input - it may be your prompt is doing something odd or incorrect. If you’re using a file, it may be the wrong or invalid file type.
  • If this is happening across a wide range of prompts, it may be something internal to Google. How often are you seeing the 500 error? 10% of the time? 50% of the time? (You can see this from the Traffic By Response Code graph on Google Cloud Console.)
7

Thanks for your reply. I’m trying to flag this issue because, despite the simplicity of the code, it still results in a 500 error. This suggests there might be an issue with the gemini-1.5-pro-latest endpoint. When using gemini-1.5-flash, everything works fine, and I’m currently using gemini-1.5-flash in production.

I am using this official DOCS Generate structured output with the Gemini API  |  Google AI for Developers