Weird 500 error when changing only the content string sent in

Gemini API
1

I have this code. The first content causes a 500 error, the 2nd “Hello” does not. Seems a bug?

        chat = orchestrator.start_chat()
        content = (
            "A user has asked a question related to Quarto. Here is what has happened so far:"
            f"<chat history>{conversation_text}</chat_history>"
            f"Please help the user with their question:<user input>{question}</user input>"
        )

        content = "Hello"

        log.info(f"# Loop [{guardrail}] - {content=}")
        response = chat.send_message([content], stream=True)

the error for the first content is:

{
  "answer": "QNA_ERROR: An error occurred while processing /vac/quarto_test: 500 An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting traceback: Traceback (most recent call last):\n  File \"/Users/mark/dev/sunholo/sunholo-py/sunholo/agents/flask/vac_routes.py\", line 206, in handle_process_vac\n    bot_output = observed_vac_interpreter(\n                 ^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/mark/dev/sunholo/quarto-agent/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py\", line 225, in sync_wrapper\n    self._handle_exception(observation, e)\n  File \"/Users/mark/dev/sunholo/quarto-agent/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py\", line 428, in _handle_exception\n    raise e\n  File \"/Users/mark/dev/sunholo/quarto-agent/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py\", line 223, in sync_wrapper\n    result = func(*args, **kwargs)\n             ^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/mark/dev/sunholo/quarto-agent/quarto/vac_service.py\", line 161, in vac\n    result = vac_stream(\n             ^^^^^^^^^^^\n  File \"/Users/mark/dev/sunholo/quarto-agent/quarto/vac_service.py\", line 52, in vac_stream\n    response = chat.send_message([content], stream=True)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/mark/dev/sunholo/quarto-agent/.venv/lib/python3.12/site-packages/google/generativeai/generative_models.py\", line 578, in send_message\n    response = self.model.generate_content(\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/mark/dev/sunholo/quarto-agent/.venv/lib/python3.12/site-packages/google/generativeai/generative_models.py\", line 325, in generate_content\n    iterator = self._client.stream_generate_content(\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/mark/dev/sunholo/quarto-agent/.venv/lib/python3.12/site-packages/google/ai/generativelanguage_v1beta/services/generative_service/client.py\", line 1122, in stream_generate_content\n    response = rpc(\n               ^^^^\n  File \"/Users/mark/dev/sunholo/quarto-agent/.venv/lib/python3.12/site-packages/google/api_core/gapic_v1/method.py\", line 131, in __call__\n    return wrapped_func(*args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/mark/dev/sunholo/quarto-agent/.venv/lib/python3.12/site-packages/google/api_core/retry/retry_unary.py\", line 293, in retry_wrapped_func\n    

return retry_target(\n           ^^^^^^^^^^^^^\n  File \"/Users/mark/dev/sunholo/quarto-agent/.venv/lib/python3.12/site-packages/google/api_core/retry/retry_unary.py\", line 153, in retry_target\n    _retry_error_helper(\n  File \"/Users/mark/dev/sunholo/quarto-agent/.venv/lib/python3.12/site-packages/google/api_core/retry/retry_base.py\", line 212, in _retry_error_helper\n    raise final_exc from source_exc\n  File \"/Users/mark/dev/sunholo/quarto-agent/.venv/lib/python3.12/site-packages/google/api_core/retry/retry_unary.py\", line 144, in retry_target\n    result = target()\n             ^^^^^^^^\n  File \"/Users/mark/dev/sunholo/quarto-agent/.venv/lib/python3.12/site-packages/google/api_core/timeout.py\", line 120, in func_with_timeout\n    return func(*args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/mark/dev/sunholo/quarto-agent/.venv/lib/python3.12/site-packages/google/api_core/grpc_helpers.py\", line 174, in error_remapped_callable\n    raise exceptions.from_grpc_error(exc) from exc\ngoogle.api_core.exceptions.

InternalServerError: 500 An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting\n"
}
2

Welcome to the forum. Not sure this will address your concern.
I would avoid using tags that include spaces; and I would make sure end tags match start tags. In the specific sample, that means changing chat history to chat_history, and user input to user_input.

Hope that helps!

3

That is a good detail to fix, thanks :). But like you said, was not the issue.

I think I found the problem: it was more about the impact of the prompt rather than the style of it - it encouraged an execution of a particular function which then caused a 500 error. Changing the prompt to induce another function and its all 200 again. I guess this has turned into a feature request to have more informative errors when sent in functions are misbehaving.

4

Here is the function that causes the API to 500, for some reason:

def quarto_render(markdown_content: str, output_format: str = "html", output_filename: str = "output.html") -> dict:
    """
    Supply the quarto markdown content to this function and it will render it and upload to a Google Cloud Storage bucket, returning the GS URI
            
    Args:
            markdown_content (str): The Quarto markdown content to render.
            output_format (str): The desired output format (e.g., "html", "pdf"). Default is "html".
            output_filename (str): The filename for the rendered output. Default is "output.html".
            
    Returns:
            dict: A dictionary with the result of the rendering process, including the URL of the uploaded file.
    """
    pass
5

And final follow up for now, it seems trying to pass around markdown as strings is generally a Bad Idea, so when I pass instead the filename location the function is all working again.

1 Like