OpenAI and xAI Complete 1000 Turns, Gemini Repeatedly Returns HTTP 500 Errors

Gemini API
1

Hello,

I’m benchmarking multiple LLM providers using exactly the same production runtime, client implementation, and benchmark scenario.

OpenAI

  • 1000/1000 turns completed successfully.

xAI Grok

  • 1000/1000 turns completed successfully.

Gemini

  • Endpoint: Google Gemini OpenAI-compatible endpoint
  • Models tested:
    • gemini-2.5-pro
    • gemini-2.5-flash

Both Gemini models repeatedly return HTTP 500 Internal Server Error during the benchmark.

Example:

[0001/1000] OK
[0002/1000] OK
[0003/1000] OK
[0004/1000] OK
[0005/1000] HTTP 500 Internal Server Error
[0006/1000] OK
[0010/1000] HTTP 500 Internal Server Error
[0015/1000] HTTP 500 Internal Server Error
[0016/1000] HTTP 500 Internal Server Error

The benchmark uses the same runtime, prompts, retrieval pipeline, and retry logic for all providers. The only variable is the underlying API provider.

The same runtime completes successfully with OpenAI and xAI. Since the only variable is the API provider, this behavior appears to be specific to the Gemini API or its OpenAI-compatible endpoint.

Has anyone experienced similar behavior, or is there a known issue affecting the OpenAI-compatible Gemini endpoint during long-running workloads?

Thank you.

2

solid benchmark setup, since everything else is held constant this does look like an intermittent reliability issue thats specific to gemini rather than your client code. worth trying the native genai sdk endpoint instead of the openai compat layer to see if the error rate changes, that would help tell whether its the compat shim or the backend itself. also worth reporting it with your request ids here: https://ai.google.dev/gemini-api/docs/troubleshooting