Sorry if this is posted in the wrong topic/forum. I have no idea where to take this question so please feel free to redirect me.
In the world of Web UI (.com / JS experiences), there is currently no out-of-box or standardized way for AI agents (like Gemini in Chrome or Copilot in Edge) to use “tools” on the site. Those tools could be access to MCP endpoints but it could also just be traditional REST data. Right now, it just kind of reads the page and spits out text. So to improve the actual digital experience, Google Chrome team has proposed Web Script Tools: script-tools/explainer.md at main · explainers-by-googlers/script-tools · GitHub
^ Which will allow browser AI agents to communicate to “tools” through the UI, thus giving the UI visibility into what’s being communicated but more importantly an opportunity to dynamically update the UI to optimize for realtime guest-AI interaction.
The real “unlock” here is that we move beyond the interaction being bound to the chat window, but finding a way for the 3P AI and 1P UI to work together.
Similarly, what does Gemini team think about enabling this type of integration between Gemini Live and native apps? For example, what if a guest opens the Target native app, then opens Gemini Live and shares their screen, then says to “make a drive up order for my usual weekly groceries and set it for 1 PM”, and instead of awkwardly trying to guess what their “usuals” are, Gemini could directly call a native “tool” exposed via the Target native app for something like “weeklyBuyItAgainGroceries()”, and the Target app UI also updates to show the drive up list being built in real time.
Is something like this already in the works in the Gemini team? Would there be interest in collaborating with Target on a use case?