[Bug Report] Severe Server-Side Implicit Cache Eviction on Gemini 4 Neon (64.5% Miss Rate at 400K–500K Context, ~95% Redundant Uncached Input)

Environment

  • Client: Google Antigravity Desktop [v2.33.0(Windows) and v2.32.2(Linux)]
  • OS: [Linux(CachyOS) / Windows]
  • Plan: Google AI Ultra (5x) — exclusive usage of the family quota pool
  • Model: Gemini 4 Neon (Extra thinking)
  • Workload: Single-threaded coding session (~50 minutes)

Summary

In a single 50-minute single-threaded session using Gemini 4 Neon (Extra), Antigravity consumed ~15% of my weekly Ultra 5x quota.

By auditing all 150 consecutive model calls in the local session logs, I confirmed that the client prefix remained byte-for-byte stable throughout the session. The massive quota burn was caused by 48 server-side implicit cache evictions (cache_read_tokens = 0), which forced 16.66M redundant historical tokens to be re-processed as uncached input (~95% of the session’s total ~17.5M uncached input).

Without server-side eviction, this entire session should have required only ~850K uncached input tokens (~20x lower).


1. Proof That Client Prefix Stability Is Intact (Not a Harness/Schema Mutation)

Comparing each turn’s input_tokens and cache_read_tokens against the previous turn’s total prompt (prev_input + prev_cache) across 150 calls:

Outcome Count Observed Delta vs. Previous Total Prompt
Cache Hit 100 cache_k - prev_total_prompt = median +1 token (min -2,049, max +1)
Full Cache Miss (cache=0) 48 input_k - prev_total_prompt = median ~1,000 tokens (new content is only 1.1% of prompt)
  • On hits, cached tokens match 100% of the previous prompt—proving that systemInstruction, tool schemas, and message history did not drift.
  • On misses, the prompt is a strict append-only superset of the previous turn (typically adding ~700–2,000 tokens of tool output).
  • Conclusion: The cache entries are being evicted/lost on the Google backend, not invalidated by client-side prefix changes.

2. Concrete Miss Examples from Session Logs

A. Back-to-back misses during standard file/build calls (Steps 136 → 138):

step 136  in=347,192  cache=0  (prev prompt: 344,438 | delta: +2,754)
step 138  in=347,935  cache=0  (prev prompt: 347,192 | delta:   +743)

Executed during dotnet restore and read_file calls with zero structural prompt changes (~695K uncached tokens wasted across just these two turns).

B. Three consecutive misses within 37 seconds (Steps 157 / 159 / 161):

Plaintext

step 157  in=378,973  cache=0  (+9,148 | edit_file)
step 159  in=381,092  cache=0  (+2,119 | dotnet build)
step 161  in=381,747  cache=0  (  +655 | read_file)

Step 161 added only 655 tokens and ran 37 seconds after the prior request, yet still suffered a complete cache miss (cache=0).

C. The only legitimate structural reset (Step 281 → 282):

Plaintext

step 281  CHECKPOINT  in=5,357   cache=511,722  (hit 512K limit, triggered compaction)
step 282              in=22,159  cache=0        (expected miss after summary replacement)

Out of 48 misses, only Step 282 was caused by an actual prefix change (compaction). The remaining 47 misses were unexpected backend evictions.

3. Strong Eviction Signature: Monotonic Scaling with Context Size & Interval

Miss rate scales monotonically with context size (excluding edge buckets):

Context Size Cache Miss Rate
100K – 200K 8.6%
200K – 300K 30.4%
300K – 400K 40.6%
400K – 500K 64.5%

Miss rate scales with inter-call interval (for context > 300K):

Inter-Call Interval Cache Miss Rate (>300K Context)
< 15s 14%
< 40s 34%
< 60s 42%
All intervals 49%
  • Poor cache rebuild reliability: Immediately following a miss, the subsequent turn only re-hit the cache 38% of the time, leading to multi-turn miss cascades (like steps 157/159/161).

  • Because normal agentic tool execution + Extra thinking routinely takes 15–60 seconds per turn, large-context sessions (300K–500K) are systematically punished by implicit cache eviction before the next turn even arrives.

4. Total Quota & Token Impact

  • 48 misses resent: 16,850,279 tokens

  • Redundant historical prefix re-billed: 16,657,578 tokens (98.9% of resent tokens, ~95% of total ~17.5M uncached input)

  • Expected uncached input without eviction: ~850,000 tokens

While the recent context window expansion to 512K is welcome, a 40%–64.5% server-side eviction rate above 300K turns long contexts into a massive quota trap rather than an advantage.

Questions / Feature Requests for the Team

  1. Is the Gemini 4 backend currently experiencing aggressive implicit KV cache eviction or non-sticky shard routing for large prefixes (200K–512K)?

  2. Can Antigravity adopt explicit session-pinned caching or sticky routing with a TTL that realistically covers normal agentic tool-execution gaps (60–120s)?

  3. When a byte-identical prefix unexpectedly returns cache_read_tokens = 0 due to backend eviction, can quota accounting protect users from being charged full uncached rates for infrastructure misses?

Need fix now >:c what am I paying $100/mo for >:c