Issues with deleting File Search Stores and indexed documents

Gemini API
, ,
1

I’m currently working on a project using the Google Gemini File Search API.

While most functions—such as creating stores, uploading documents, and generating RAG-based answers—work smoothly, I’n encountering significant performance issues and errors when managing a large volume of indexed documents (over 1,000 files).

I would like to seek advice on the following two issues:

1. Performance bottleneck in locating and deleting specific files The current API doesn’t seem to support filtering the document list by ID or filename within a File Search Store. To delete a specific file, I have to fetch the entire paginated list and iterate through it, which is extremely slow when the store contains many documents. Is there a more efficient way to target and delete a single file without listing everything?

2. 503 Service Unavailable error when deleting a File Search Store When I try to delete an entire File Search Store that contains a large number of documents, the API often returns a 503 Service Unavailable error. This happens even when I set the force parameter to true. It appears the system is struggling to handle the bulk deletion of the embedded data.
While I could manually implement a batch deletion process for every document in the store, I’m hoping for a more inherently efficient method or a specific API capability designed for large-scale cleanup

Error Log:

INFO:httpx:HTTP Request: DELETE https://generativelanguage.googleapis.com/v1beta/fileSearchStores/[STORE_ID]?force=True "HTTP/1.1 503 Service Unavailable"

ERROR:app.services.gemini_service:Error deleting file search store: 503 UNAVAILABLE. 
{'error': {'code': 503, 'message': 'The service is currently unavailable.', 'status': 'UNAVAILABLE'}}

Questions:

  • Are there any best practices for managing or deleting specific files in a large-scale File Search Store?

  • How can I reliably delete a store with many documents without triggering a 503 error? Is there a recommended cleanup procedure for large stores?

I would appreciate any insights or workarounds. Thank you!

3

Hi @Daze, welcome to the community!

Currently, the supported parameters for query are pageSize and pageToken; hence, you can’t filter it using name.

You can try to maintain a lightweight local mapping that links your internal filename/ID to the Gemini Resource ID. When you upload, you immediately save the pair; when you want to delete, query your local DB to get the resource ID and then call delete().

When you use force=True on a store with 1000+ files, the backend attempts to perform an elaborated delete. If this exceeds the standard HTTP timeout, you get the 503 error.

You can try to implement exponential backoff logic for this.

You can implement it by progressively increasing the wait time between retries. This documentation provides specific examples in our Python error handling cookbook

Thank you!