RAG with Gemini

Gemini API
1

Hi! I am trying to design an app which answer to Gemini and Gemini has to search into a set of data in JSONL format. As far as I know I can do this perfectly with Vertex AI Search, but I can not get it working. Perhaps the prompt I am using is a bit complicated. I will explain the case better:

The JSONL has a list of elements with some keywords. Each one of this element represent a pictogram (an image). I.e:

{"_id": 2248, "categories": ["beverage", "mineral rich food"], "keywords": [{"type": 2, "keyword": "water", "hasLocution": true, "plural": "waters"}], "tags": ["feeding", "food", "beverage", "mineral rich food"]}

I want Gemini choose a series of pictograms (from this JSONL) which could be used to answer a question.

For instance, with this prompt: "What's the weather like today?"
It should answer something like:

[
  { "_id": 2796, "keyword": "Sunny" },
  { "_id": 2779, "keyword": "Windy" },
  { "_id": 2936, "keyword": "Rainy" },
  { "_id": 2884, "keyword": "Snowy" },
  { "_id": 2814, "keyword": "Stormy" },
  { "_id": 39218, "keyword": "Partly cloudy" },
  { "_id": 2885, "keyword": "Foggy" },
  { "_id": 2986, "keyword": "Rainbow" }
]

But I can not get it working. However in Gemini app is working perfectly, but I have to include the JSON into the prompt each time (4MB). In the app I am using a prompt like:

You have to select a set of pictograms to answer a question. 
The pictograms are in this url: https://blablalbala
The question is: What's the weather like today?

And Gemini returns some good pictograms from the JSONL in the url.

What I missing?

Is there any better approach than Vertex AI Search for this goal?

Thanks

1 Like