Forcing Gemini Live (WebSocket) to Always Use Specific Documents

Gemini API
, ,
1

I am building an assistant with Gemini Live using the WebSocket API. I need some help understanding how to guarantee Gemini responds using context from a specific document.

In my setup, I know exactly which page the user is currently on, and therefore I know exactly which document(s) should be used when answering user’s questions. I need a way to have Gemini, 100% of the time, without fail, always refer to predetermined documents every single time Gemini responds to the user.

Currently with my RAG application, even with strict prompting, Gemini sometimes correctly recalls information from my documents, but also often states it does not have access to the information, even when testing on the same page with the same or similar questions.

Please note, we have many different pages, each with its own corresponding documents.

I have been looking through documentation for the Live API, especially with RAG use, but am struggling to find a solution that solves my problem.

Any assistance would be greatly appreciated.

3

Anyone able to help?

5

Hi @RileyOlson,
Welcome to AI Forum! Thanks for reporting this issue.
There are two primary approaches for creating a RAG system:

  • File Search API: The Gemini API provides a File Search tool that allows you to build a RAG system; however, please note that the File Search tool is not supported with Live API models.

  • RAG Engine: This is a Vertex AI service that allows you to utilize the Live API for RAG systems.