Reading throught the API docs for gemini live, it says that you can update the session parameters live without having the restart the session
You can change the configuration parameters except the model during the session.
I want to use this to edit the system prompt and available tools without having to restart the session
However looking through the JS and Python SDK docs, I can’t figure out how to do this. It looks like LiveConnectConfig is only ever used inside the connect method?
I see that I have access to the websocket connection under the conn property, but I’m struggling to understand via the docs what I’m supposed to send
The docs say that BidiGenerateContentClientMessage should be the first and only in the first message so I don’t understand how and where to do this. It is also not possible to do this in the studio web UI
Ah thank you!
So does this mean that using the session resumption token I can .stop() the previous session, then use the token to resume the session with a modified system prompt?
You currently cannot update system instructions during an active session because of websocket infrastructure limitations. The only way to update them is by resuming a session after approximately 10 minutes of inactivity. While the team is working on enabling real-time changes, this functionality isn’t available yet.
@GUNAND_MAYANGLAMBAM It’s been a while and I was just wondering if a way to update system instructions midway has been incorporated since your last response
Also Like @Ayesha_Saleem has said in the vertex Ai documentation it clearly says that system prompt can be updated. So which one is correct?
I’m looking to do something similar and have the tools available mid-session.
I’m working on a large scale app with countless possible tools and I don’t want to front-load them all, so I’m looking for the ability to update a ‘store’ as needed with the available tools for the AI.
This ‘dynamic tool selection based on conversation context’ seems to also be referenced and recommended in the function calling best practices, but there is nothing about HOW to actually do this:
I’m looking for something related to OpenAI’s session.update functionality where we can update the list of tools (and potentially ‘data’) to the LiveAPI realtime model.
However, I cannot find anything like that.
I did notice lately that the LiveAPI realtime native-audio model has updated rate limits which seems much more generous that previous, I guess because this will allow us for now to just break the connection and reconnect more frequently and update the front-loaded tools, this is not ‘ideal’ but this at least remedies this for some time.
Looking at the TypeScript definition file for the Google Gemini Live API’s genai module, I can confirm that there is no functionality similar to OpenAI’s session.update for dynamically updating tools/functions during an active session.
Based on my examination of the code, here’s what I found:
Key Findings:
Tools are set at connection time: In LiveConnectParameters, tools are specified in the config object when creating the session:
typescript
Setup message is sent once: The LiveClientSetup interface (which contains tools configuration) is described as:
“Message to be sent by the system when connecting to the API. SDK users should not send this message.”
This is only sent during the initial connection establishment.
No update mechanism: There are no methods like:
◦ session.updateTools()
◦ session.updateConfig()
◦ session.update()
The Session class only has these methods:
• sendClientContent()
• sendRealtimeInput()
• sendToolResponse()
• close()
Conclusion:
You are correct - tools must be “front-loaded” at session creation time. If you need to change the available tools, you would need to:
Close the current session
Create a new session with the updated tool list
This is a significant architectural difference from OpenAI’s Realtime API, which allows mid-session updates to the tool configuration.
Intra-app navigation really needs the ability to update the context, and being able to narrow it helps as well, since it can be adjusted to match what the user sees at the moment, enhancing its usefulness and pertinence. I don’t want to have reasons to use OpenAI instead, but this seems to be one!
