Unreliability of Gemini API - Error while creating cache

Gemini API
25

Is the fix live on prod? It is taking light years, considering Gemini is Google’s flagship product.

27

Hey guys, the issue seems to be resolved now!

28

Thanks for your patience on this everyone, the fix is live. Please let me know if y’all run into any other issues.

29

Just tested, working for me! Thanks for the hard work @Vishal and team!

1 Like
31

Well , Guys…
It’s back again

33

This is also happening to me again. Very frustrating. @Vishal Can you please flag this!?

34

Hey @Jack_LV - can you share what model you’re using & the data (e.g. pdf, video, audio) you’re trying to cache?

35

Hi @Vishal, thanks for the reply. The data are JPEGs, each corresponding to one page of a PDF scan of an old document. They’re ~3MB each.

The problem arose when I was using gemini-1.5-pro-002. I get the same 403 error when using 1.5-flash-002. However, when I use either gemini-1.5-pro-001 or gemini-1.5-flash-001 with identical code (posted below), I get “InvalidArgument: 400 Cached content is too small. total_token_count=2622, min_total_token_count=32768,” etc. Very bizarre.

Here’s the code, which fails at the caching.CachedContent.create() call:


   while pic.state.name == 'PROCESSING':
       print('Waiting for text to be processed.')  
       time.time.sleep(1)
       pic = genai.get_file(pic.name)
   print(f'Picture uploaded: {pic.uri}')

   cache = caching.CachedContent.create(
   model='models/gemini-1.5-pro-001',
       display_name=f'ocr_test', # used to identify the cache
       system_instruction=(
           """
           You are a research assistant to a historian helping to digitize scanned archival documents. 
           """
       ),
       contents=[pic],
       ttl=datetime.timedelta(minutes=20)
   )

   model_cached = genai.GenerativeModel.from_cached_content(
           cached_content=cache,
           generation_config = {
           "temperature":0,
           "max_output_tokens":8192
       }
   )```
36

Thanks for these details, @Jack_LV. Looking into it now

37

@Jack_LV - this should be fixed now! Let me know if you’re still running into any issues

38

Thanks @Vishal it is working!

41

I got this error many times in the past few days including today.
Error: 403 You do not have permission to access the File xxxx or it may not exist.

I am using gemini-2.0-flash-exp with files uploaded with genai.upload_file().

The error occurs more or less randomly. It seems to happen when I open my app in multiple browers and/or chat for several rounds. And it seems to occur in Safari on iphone more often other browers. But I am not positive.

It is quite confusing and annoying that this problem still exists. It would heavily affect Gemini’s usability and adoption. Please check and fix the problem. thank you!