Using Grounding with Apps Script

Google AI Studio
,
1

We’ve been using the gemini-1.5-flash-8b model to summarise a website by its HTML, but now grounding is available we were testing to switch to just the URL. The testing we have done in Google AI Studio looks really good, but when we implement this via accessing the API using Apps Script the results are completely different. Here’s the payload JSON we’re submitting to the v1beta URL and its results are not the same as the same settings in Google Ai Studio - are we doing something wrong?

‘{“system_instruction”:{“parts”:{“text”:“systemInstructions”}},“contents”:[{“parts”:[{“text”:“1. Create a summary of the organisation from the website at the URL included below. \n2. Include what kind of organisation it is, what it does, and, ideally, try and include how the organisation likes to work.\n3. If possible provide the address of the headquarters of the organisation. \n4. Never mention anything about the website, e.g. never put in the response \“The provided website uses WordPress\” or similar text, only refer to the organisation.\nHere is the URL:\nhttps://www.steegle.com”}]}],“tools”:[{“google_search_retrieval”:{“dynamic_retrieval_config”:{“mode”:“MODE_DYNAMIC”,“dynamic_threshold”:0.4}}}]}’

1 Like
3

Hi @StephenHind , Welcome to the forum.

Did you try with 0.3 dynamic_threshold in SDK?? And check whether the result are still different.

5

Unfortunately that’s not a valid test because nearly every time I run the prompt with the exact same settings I get different responses, so there’s no way to tell if I change the dynamic_threshold if the difference is from that change or Gemini’s random variability.

What’s the correct way to check if this is working? The answer received from the API is nothing similar to AI studio, so how are we meant to use AI Studio to test what works in the API?

Is there an official support channel where we can get support as we’re paying to use the API?

6

Hi @StephenHind
On reading your scenario, I have just a small question. What is your temperature, topK and topP settings? Are these settings the same in your app script and in Google AI Studio?
Hope this helps and would love to know the outcome of this issue.
Regards

8

So, I tried testing with temperature of 0.1 to make the model response more deterministic, and set the dynamic_threshold to 0.3 in both AI Studio and the REST API. I observed that the model’s responses were more or less the same with the same prompt you have provided. Below is the request I used:

import requests
import json

prompt = """
1. Create a summary of the organisation from the website at the URL included below. \n2. Include what kind of organisation it is, what it does, and, ideally, try and include how the organisation likes to work.\n3. If possible provide the address of the headquarters of the organisation. \n4. Never mention anything about the website, e.g. never put in the response \“The provided website uses WordPress\” or similar text, only refer to the organisation.Here is the URL: https://www.steegle.com"""

# Define the API endpoint
api_url = "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-8b:generateContent?key=GEMINI_API_KEY"

payload = {
    "contents": [
        {
            "parts": [
                {"text": prompt}
            ]
        }
    ],
    "generationConfig": {
            "temperature": 0.1
        },
    "tools": [
        {
            "google_search_retrieval": {
                "dynamic_retrieval_config": {
                    "mode": "MODE_DYNAMIC",
                    "dynamic_threshold": 0.3
                }
            }
        }
    ]
}

headers = {
    "Content-Type": "application/json"
}


response = requests.post(api_url, headers=headers, json=payload)


if response.status_code == 200:
    response_data = response.json()
    print(json.dumps(response_data, indent=2))
else:
    print(f"Error: {response.status_code} - {response.text}")

And one more thing, when you use the grounding feature, the model doesn’t have the capability to open/process the provided URL endpoint in the prompt. Based on your prompt, the Gemini API returns grounding sources along with the response content.

9

The response I get is purely text: there are no sources mentioned at all; where do I find the sources?

PS the URL was to help find the site in the grounding: I have another version that gets the HTML of the page and submits that instead, but then the grounding doesn’t really make any different as it only looks at the HTML provided.

1 Like
10

You should be able to find the sources in groundingMetadata. Please refer the colab gist link below.

Grounding

1 Like
12

Yes I have the groundingMetadata and the renderedContent is quite different to the plain text response given, so I will have to think how I can utilise that instead.

@Mannetjie_Van_Rooyen thanks for the heads up on the other parameters: I will experiment with those and see if I can get improvements (I’ve read the docs but still not entirely sure what topP is meant to do ; - )