Constant 500s on Gemma 4

Gemma
41

Feels like errors are back with the same error rate, around week ago everything was okay and error rate was around 2-5%, and then it’s again 70% error rate

42

I really think that infinite TPM for gemma 4 isn’t the best idea for free tier. As somebody may use it. Like 256k random tokens per request in 15 requests per minute sounds not very good, it’s 3.8 million tokens per minute that might be not even be cached in some situations

43

Looks like errors are back, been experiencing them for few hours already, error rate is 70%

44

It’s really annoying, I’m hitting rate limits when I’m not supposed to, because errors 500 are counted into the usage limit, so I don’t have full amount of requests

45

It’s literally 1.5k RPD, how are you hitting limits?

46

With errors it becomes 450 requests (error rate 70%, and errors are counted into the usage limit), and I use it for different purposes. Without errors I don’t hit the rate limit

47

I use it like for coding help (not creating code, more like helping with docs), translating, etc. So hitting rate limit isn’t that hard when errors are counted to the limit… And it’s so annoying to have that big error rate, I simply cannot use gemma…

48

For me error are instant and they are now 95%. I use to review the specs and review file by file. It’s not reliable and can’t be used to do anything.

I think this is because of the resources Google allow for Google AI studio api.

Same model work on vertex without any issues because it’s paid but API endpoint is different too.

It might be possible that vertex API has priority over Google AI studio or Google only allocate resources when vertex load is down under some percentage. If vertex load is high and around certain percent Google AI studio api hit 500 regularly.

49

Yeah, today error rate was crazy, just impossible to use gemma

50

I really hope for fix, it just became unusable

51

From last 2-3 hours it’s working and almost 95% success rate. I am in UK and it’s Sunday night so no load on servers so we have capacity on AI studio. I will see how it will go tomorrow when everyone will be back to use other models.

52

Errors are back on as Monday afternoon and evening as usage is back.

53

It’s really random, sometimes I have 16% success rate, sometimes it’s 95%, one day it even was 3%

54

at least you had a success rate over 10 % :frowning: I had less than that and I have tried it with different devices and accounts and no luck for me.

55 
curl -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemma-4-26b-a4b-it:streamGenerateContent?alt=sse" \
     -H "x-goog-api-key: $GEMINI_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "gemma-4-26b-a4b-it",
       "contents": [
         {
           "role": "user",
           "parts": [
             {
               "text": "Ansver 1+1 only."
             }
           ]
         }
       ]
     }'

quite often it reports 500:

{
  "error": {
    "code": 500,
    "message": "Internal error encountered.",
    "status": "INTERNAL"
  }
}

I vibe-coded a script to do some benchmarks, the quality is horrible. I can accept the 429 but not the spam of 500s

================================================================================
🚀 GEMINI BENCHMARK LIVE DASHBOARD (Total per model: 100)
================================================================================

📊 MODEL: [gemma-4-26b-a4b-it]
--------------------------------------------------------------------------------
 Progress:      [56/100] | Success Rate: 51%
 Results:       ✅ Success: 51 | ❌ Hard Failures: 5
 Live State:    ⚡ Active Workers: 34 | 🕒 Backoff/Waiting: 10
 Retries Logs:  🔄 Total Retries: 329 | ⏳ Avg Wait Time (Backoff): 74.7s
 HTTP Errors:   💥 Internal (500): 96 | 🚦 Rate-limit (429): 634 | ❓ Other: 0
 Latency:       ⏱ Avg Latency (Success): 368184ms
 Latest Logs:   #9: Waiting HTTP 500 (Next retry in 120.0s) | #19: Waiting HTTP 500 (Next retry in 120.0s)
--------------------------------------------------------------------------------

📊 MODEL: [gemma-4-31b-it]
--------------------------------------------------------------------------------
 Progress:      [98/100] | Success Rate: 92%
 Results:       ✅ Success: 92 | ❌ Hard Failures: 6
 Live State:    ⚡ Active Workers: 0 | 🕒 Backoff/Waiting: 2
 Retries Logs:  🔄 Total Retries: 555 | ⏳ Avg Wait Time (Backoff): 23.3s
 HTTP Errors:   💥 Internal (500): 14 | 🚦 Rate-limit (429): 561 | ❓ Other: 0
 Latency:       ⏱ Avg Latency (Success): 257305ms
 Latest Logs:   #44: Waiting HTTP 500 (Next retry in 120.0s) | #72: Waiting HTTP 500 (Next retry in 120.0s)
--------------------------------------------------------------------------------
56

The final result:

📊 MODEL: [gemma-4-26b-a4b-it]
--------------------------------------------------------------------------------
 Progress:      [100/100] | Success Rate: 75%
 Results:       ✅ Success: 75 | ❌ Hard Failures: 25
 Live State:    ⚡ Active Workers: 0 | 🕒 Backoff/Waiting: 0
 Retries Logs:  🔄 Total Retries: 746 | ⏳ Avg Wait Time (Backoff): 35.5s
 HTTP Errors:   💥 Internal (500): 112 | 🚦 Rate-limit (429): 634 | ❓ Other: 0
 Latency:       ⏱ Avg Latency (Success): 518941ms
 Latest Logs:   All workers clear or completed.
--------------------------------------------------------------------------------

📊 MODEL: [gemma-4-31b-it]
--------------------------------------------------------------------------------
 Progress:      [100/100] | Success Rate: 92%
 Results:       ✅ Success: 92 | ❌ Hard Failures: 8
 Live State:    ⚡ Active Workers: 0 | 🕒 Backoff/Waiting: 0
 Retries Logs:  🔄 Total Retries: 575 | ⏳ Avg Wait Time (Backoff): 22.4s
 HTTP Errors:   💥 Internal (500): 14 | 🚦 Rate-limit (429): 561 | ❓ Other: 0
 Latency:       ⏱ Avg Latency (Success): 257305ms
 Latest Logs:   All workers clear or completed.
--------------------------------------------------------------------------------