I have a Conversational Agent (Gemini 2.5 Flash) that is instructed via Playbook to always query a tool Datastore of a website before formulating a response. We’re seeing ~5-10% of questions answered directly without using the tool, often resulting in hallucinations of non-existent webpage URLs, email addresses, and generally inaccurate information. Responses utilizing the tool typically have very high accuracy. Does anyone have any recommendations for better enforcement of tool use?
Relevant Assistant Playbook Prompt:
Information Retrieval Protocol
- MANDATORY: For ALL questions:
- 1. Search ${TOOL:name} FIRST
- 2. Wait for search results before formulating response
- 3. Base primary answer on website data