Constant timeout errors when calling gemini

Gemini API
,
1

Context -

  1. We make high concurrent api calls of almost 500 - 600
  2. We are not reaching the token limit or request limit

Issues -

  1. Constant timeout errors when calling azure ( openai ) & Gemini
  2. LIbraries used - agents SDK by openai, ADK by Google gemini
  3. Tried different models, still same result.
  4. Tried with different connect timeouts, still same result.
3

Hi @sprout_head,

Welcome to the Google AI Forum! :confetti_ball: :confetti_ball:

The fact that all services are failing specifies that there is an issue on your end.. Timeout error happens for multiple reasons.

  1. Networking bottlenecks: When Temporary Ephemeral ports your OS assigns when making outbound connections don’t get closed in time can cause this issue.
  2. Firewall/Proxy/Load Balancer Timeouts: If your requests pass through an internal firewall, proxy, or load balancer, these devices often have their own default idle connection timeouts.
  3. Resource Exhaustion on Your Server: High CPU can cause requests to be processed slowly on your end
  4. Code/Application Specific Issues: Check if there any blocking operations in your code path that get executed for each concurrent request such as Database calls, local file I/O, etc.

Please identify which issue is being caused and try again.

Happy Coding :slight_smile: