Handling Multiple PDF Files with Gemini API and Token Limit Issues

Gemini API
, ,
1

Hello,

I am currently using OpenAI GPT-4 model, where I upload over 100 PDF files to a vector database, and I can ask questions based on the content from those PDFs. However, when using the Gemini API, I face token limit errors and long response times after uploading the files. Is there a solution for this? Is it possible to efficiently process and query multiple PDF files with Gemini API, extracting information from them without hitting token limits or facing long delays?

I would appreciate any help or suggestions!

1 Like
4

Hi @bykemalh

Welcome to Forum!
The Gemini API has certain limitations when it comes to uploading files. For detailed information, refer to this link.

Here are a couple of solutions that might help in your scenario:

Solution 1: Summarize the Files

You can summarize each PDF and upload the summaries of all 100 files to Gemini Pro, leveraging its 2-million token limit. This approach allows you to process large volumes of content while staying within the API constraints. Make Sure that the summaries retain the most critical information to prevent loss of context or key details.

Solution 2: Chunk the Files

If you don’t need to process all the files simultaneously or if the files are independent, you can divide them into smaller chunks. These chunks can then be processed individually using the Gemini API. This method helps manage token limits effectively.

Thanks

6

The first approach has a critical drawback regarding response time. When I send a large amount of data, the response time can stretch to 5 minutes or more, which is impractical.

The second approach, on the other hand, is not suitable for my needs. I’m developing a chatbot that must have complete knowledge of all company-related information, and this method doesn’t align well with that requirement.

What I aim to do is use OpenAI’s vector database to tune the model with company-specific PDFs. This will allow the chatbot to access company information quickly and efficiently.

8

@bykemalh

You can utilize the text-embeddings-004 model available in the Gemini API to store company-specific PDFs. Refer to this notebook for guidance on using text-embeddings-004 with ChromaDB.

Thanks