Implicit Caching: Gemini 2.5 Pro Preview 05-06

Gemini API
,
1

Anyone get implicit caching results with their prompts yet? I still haven’t seen my cached tokens in my usage metadata change from None. I had a 54,000 token static prompt, sent the same message several times, not a single token cached.

I dropped the prompt down to 11,000 tokens, continued sending the same message over and over, still None.

cache_tokens_details=None cached_content_token_count=None candidates_token_count=74 candidates_tokens_details=None prompt_token_count=11629 prompt_tokens_details=[ModalityTokenCount(modality=<MediaModality.TEXT: ‘TEXT’>, token_count=11629)] thoughts_token_count=260 tool_use_prompt_token_count=None tool_use_prompt_tokens_details=None total_token_count=11963 traffic_type=None

1 Like
3

Hi @roteck, Thanks for reporting this issue. while reproducing the issue we have observed the same. Will bring this to the engineering teams attention. Thank You.

1 Like
5

You’re welcome, I was very excited to learn that implicit caching came to Gemini.

If it helps I’ve managed to successfully get implicit token caching to mostly reliably happen. The only way I’ve had success is to rapidly send message after message, then it will cache almost always with the 2nd message and on. But it seems that if I hesitate for at least 10 seconds, it doesn’t cache. Very short TTL? This makes implicit caching for my applications ineffective.

Since then I worked on learning about explicit caching and have begun using that for now until the new implicit system is working reliably.

If you need any info from me or want me to try any techniques, let me know.