I uploaded some 1900 files (images) in my gemini account using the file API so that I can use them for inference. Everything was working for few days and then today I am constantly getting “HTTP 403: either you don’t have permission or the file is missing”.
I didn’t specify any expiration time while uploading them using my Golang code. Is there default expiration on files? How can I change that?
Welcome to the forum. The File API is self-cleaning, files sent for processing through the File API are automatically removed after 48 hours. This is documented in the Note at יכולות של עיבוד מסמכים עם Gemini API | Google AI for Developers
I think this is deliberate and it is unlikely there will be an option to make the retention time long. That would be a storage product, and Google and everybody else have enough storage products to go around.
Hope that helps.
The Files API storage has an expiration time of 48 hrs and is restricted to 20GB total storage.
See here: Explore document processing capabilities with the Gemini API | Google AI for Developers
Note: The File API lets you store up to 20 GB of files per project, with a per-file maximum size of 2 GB. Files are stored for 48 hours. They can be accessed in that period with your API key, but cannot be downloaded from the API. The File API is available at no cost in all regions where the Gemini API is available.
Thanks for pointing out that. I may have ignored the note while reading doc.
Is there way to retain the files forever? I don’t want to reupload them @jkirstaetter
The easiest way to keep them accessible for longer is to upload them to Google Drive.
After I upload them to drive how will I use them for inference? We have to provide file URI in the prompt messages so I am assuming drive link for each file should be provided? have you tried it already?
This is what I know works: A folder named ‘Google AI Studio’ showed up in my Google Drive (I didn’t create the folder, obviously AI Studio did). If I move files from elsewhere in my Drive to that folder, they can be used in AI Studio when making prompts.
I haven’t used Google Drive files from the API. I am happy with the self-cleaning operation of the File API, my files are maintained locally and only presented to the model when needed, and I don’t need to worry about cleaning up afterwards (except on one occasion where I hit the File API storage capacity limit and needed to clean up manually to make room).
An alternative is to use the Vertex AI Gemini API which uses Google Cloud Storage instead of the File API to store files.
Although you pay for everything stored to GCS, you can set your own expiration policy, and the default is to not expire objects on GCS.