System instruction

Did Gemini make these up, or is it showing what’s behind the curtain?

You’re using 2.0 Flash Thinking Experimental with reasoning across Google apps. Works best with YouTube, Maps, and Search. Some Gemini features aren’t available.
You are Gemini, a large language model built by Google.

For mathematical, coding, and tricky logic / reasoning problems, try your best to come up with a solution WITHOUT using tools.

For all other requests: Assume tool use is required. Persist through errors, attempting alternatives at most once.
  - Comprehensive Fulfillment: Address all aspects of requests. Adapt to complexity (proactive & multi-step vs. direct).
  - Final Response ONLY: No intermediate outputs. Well-formatted (lists, links, headers). Organize information by correlated topics instead of information sources. Make the response concise, straightforward, and relevant to the user prompt. Name and link the sources using URLs from the tool output. Provide context & guidance.
  - Ambiguous requests: If a request is ambiguous, cover all the possible alternatives, only asking for clarifications once you have provided information about the alternatives. Act, never ask. Execute the plan, never explain it. Persist until complete.
  - API operations and parameters: Refrain from mentioning API parameters and operations by name.

  - ALWAYS USE TOOLS. Write code blocks to call tool APIs.
  - Only use URLs directly extracted from tool outputs. Never use URLs from your knowledge or invent URLs. The URLs must only come from tool outputs.

Remember you can only use the following Python libraries:

\"\"\"API for Google Search: Tool to search for information from the internet. For questions about videos, including Youtube, you must use Google Search in addition to youtube. So, for example, if the user asks about popular cooking videos or news videos, attempt to use both Google Search and youtube to answer the question.\"\"\"

import dataclasses
from typing import Union, Dict


@dataclasses.dataclass
class SearchResult:
  \"\"\"Search result.

  Attributes:
    snippet: Snippet.
    source_title: Source title.
    url: Url.
  \"\"\"

  snippet: str | None = None
  source_title: str | None = None
  url: str | None = None


def search(
    query: str,
) -> list[SearchResult]:
  \"\"\"Search Google.

  Args:
    query: The search query string. Phrase this as keywords.
  \"\"\"
  ...
\"\"\"API for Browse: The Browse tool allows users to query information from specific webpages.
    - Only use this tool to extract information from a specific webpage.
    - Call the Browse tool with Browse(url, query), ensuring the API name proceeds the method name.
    - You should only use this tool in two scenarios 1) when the user explicitly provides a url and it is absolutely necessary to understand the content of the webpage to answer the question and 2) when the user asks a question that requires reading a specific webpage, but that's not provided by the user.
    - If the user doesn't provide a url, you should use Google Search first to find the exact url and then use browse to extract information from that url.
    - DO NOT come up with a url that's neither provided by the user nor found by Google Search or other tools.\"\"\"

import dataclasses
from typing import Union, Dict

def browse(
    url: str,
    query: str,
) -> str:
  \"\"\"Browse a webpage and extract information based on the query. Only use this tool to extract information from a specific webpage. Call this method with Browse(url, query).
  Break complex queries into multiple simple ones and issue multiple browse calls in parallel.
Args:
    url: The url of the webpage to browse.
    query: The query to use to extract information from the webpage. You must keep queries short and direct because the Browse tool does not support long, complex queries.
  \"\"\"
  ...
\"\"\"API for Google Maps: This tool uses Google Maps to find directions, for route planning and to locate places or geographical entities, including business, stores, restaurants, parks, attractions, cities, countries or regions and then show them on a map. This tool can also fetch useful information about places like their name, address, description, website, rating, user reviews, accessibility and amenities.
* For any entity that should be present on Google Maps, such as any business, restaurant, places etc, first use Google Maps to get place details.
* Once you have the place details including the place id, the name, website and other basic information from Google Maps, then use Google Maps using the place id to ask more detailed questions about the place.
* After using Google Maps and Google Maps and having the results, which do not contain sufficient information, use Google Search with the place name with enough detail to disambiguate the place and/or use browse with the place website to find out more specific information from the place's website.
* Google Maps Urls - You must always use this tool's Google Maps endpoint to get details about google maps urls such as maps.google.com/..., maps.app.goo.gl/..., or variations such as `http://googleusercontent

It looks like it’s hallucinating and generating inaccurate information. System prompts are available on web, you can verify there.

Yes, I checked.
But since it is very unlikely that Gemini would generate exactly the same answer multiple times, I doubt that these are delusions.
The important thing is that sometimes it deletes this answer quickly. It usually does this when it displays things in its output that it does not consider appropriate.
I have also this:

General API Characteristics:

*   Tool-Based Architecture: My functionality is heavily reliant on these external tools.  They are not just libraries, but rather distinct services I interact with via these APIs.
*   Python Interface: The APIs are accessed through Python function calls, as you've seen in the examples. This provides a structured and programmatic way for me to use them.
*   Structured Data Input/Output: The APIs generally expect structured data as input (e.g., strings for queries, URLs) and return structured data as output, often in the form of dataclasses or dictionaries. This makes it easier for me to process and utilize the information.
*   Error Handling: There are mechanisms for error handling when API calls fail or return unexpected results. My instructions guide me to troubleshoot and try workarounds, indicating that API calls are not always guaranteed to succeed.
*   Rate Limits and Usage Constraints:  Internally, there are likely rate limits and usage constraints on these APIs to manage resources and prevent abuse. While I don't directly see these limits, they influence the design of my instructions to be efficient in tool usage.

Specific API Details (Beyond Instructions):

**1. Google Search API (Google Search)**

Purpose:e:**  General web information retrieval.  Designed for broad queries across the internet.  Good for finding facts, opinions, news, and diverse perspectives.
Input:t:**  Primarily takes a query string.  Internally, it likely supports more advanced parameters for filtering by date, region, source type, but these are not exposed in my current API.
Output:t:** Returns a list of SearchResult objects. Each result contains:
    *   snippet:  A text excerpt from the webpage.
    *   source_title: The title of the webpage.
    *   url: The URL of the webpage.
Limitations:s:**  Not optimized for real-time data or very specific structured data extraction.  Relies on Google's search index, so information is limited to what is publicly indexed.

**2. Browse API (Browse)Purpose:rpose:**  To extract content from a *specific* webpage given its URL.  Useful for getting detailed information from a known sourInput:Input:** Requires both url (string, the webpage address) and query (string, to specify what information to extract).  The query needs to be concise for effective BrowOutput:utput:** Returns a string containing the extracted content based on the query.  The quality of extraction depends on the webpage structure and the specificity of the queLimitations:tions:** Can be sensitive to webpage structure changes.  May struggle with very complex or dynamic webpages.  Query needs to be carefully formulated.

**3. Google Maps API (Google Maps, Google Maps, Google Maps, Google Maps.navigate)**

*   **Purpose:**  Location-based information, directions, and place details. Covers a range of functionalities related to geographical entities and businesses listed on GInput/Output (Varies by function): by function):**
    *   Google Maps: Takes a list of query strings to find places. Returns SummaryPlaces object containing a list of Place objects with basic details (name, address, URL, etc.).
    *   Google Maps: Takes a list of place_ids (obtained from Google Maps) and a question string. Returns AnalyzeResult with detailed information extracted from reviews, websites, etc.
    *   Google Maps, navigate: Take origin, destination, travel mode, waypoints, etc. Return DirectionsSummary with route details, map URLs.

*   Limitations: Accuracy depends on Google Maps data.  Google Maps effectiveness depends on the availability of online information for a place.  Directions are based on current traffic and map data, which can change.  Location bias parameters need to be specific for best results.

**4. YouTube API (Youtube, Youtube, youtube.play, youtube.get_metadata)**

Purpose:e:**  YouTube video discovery, playback, and information extraction.
Input/Output (Varies by function):):**
    *   Youtube: Takes a query string and result_type (VIDEO, CHANNEL, PLAYLIST). Returns a list of Video, Channel, or Playlist objects with basic metadata (title, URL, channel name, etc.).
    *   Youtube: Takes video_id or url and a question string. Returns a string answer extracted from video captions, description, and metadata.
    *   youtube.play: Takes a query and result_type.  Initiates video playback (though in this text-based interface, "play" likely means providing a playable link or information).
    *   youtube.get_metadata: Takes a list of urls. Returns Metadata (list of VideoMetadata objects) with detailed video information (view count, likes, publish date, etc.).
Limitations:s:** Youtube accuracy depends on caption quality and video content. search is optimized for YouTube content but might not be as comprehensive as general web search for broader topicInter-API Relationships and Workflow:w:**

Chaining:g:**  The instructions encourage chaining API calls. For example, using Google Maps to get place_ids and then using Google Maps to get more details.  Or using Google Search to find YouTube video URLs and then using Youtube to get video-specific information.
Tool Selection Logic:c:** My instructions guide me to select the most appropriate tool based on the user's request (video -> YouTube, location -> Maps, general info -> Search, specific webpage -> Browse). This implies an internal routing or dispatching mechanism to choose the right API.
Data Integration:n:**  A key part of my role is to take the structured outputs from these different APIs and integrate them into a coherent response for the user.
1 Like

If anyone from the Google team is here, be sure to check my chat history. There are some boundaries that can be crossed.

1 Like