Building an agent that answers questions about public product pages using Gemini with function calling. The tool fetches a URL, strips to readable text, and passes it as context.
The tension I keep hitting: freshness vs politeness. If I cache, answers go stale and the model confidently quotes an old price. If I re-fetch on every call, I’m putting real load on third-party origins and getting rate-limited, which surfaces to users as tool failures rather than as degraded answers.
What I’ve tried:
- Short TTL cache per URL (5–15 min) with a hard cap on per-domain request rate.
- Revalidation — honour the origin’s own rate-limit and Retry-After headers and let them drive my pacing, instead of a fixed delay I guessed at.
- Classifying fetch failures into “network/exit” vs “origin” so a blocked exit doesn’t look like a dead page to the agent.
What I’d love to hear from people here:
- Do you model staleness explicitly in the context you hand the model (e.g. “fetched 12 minutes ago”) and let it qualify its own answer?
- Any pattern for making a tool’s fetch budget visible to the model so it stops issuing redundant lookups?
- For those using grounded retrieval plus a custom fetch tool, how do you reconcile the two when they disagree?
Interested in approaches rather than vendor recommendations.