To give some context (no pun intended!), this is for a chat app that is getting stock info like quantity/prices and putting it in the system_information correctly and all works well until the system_information changes (as the qty/price will), once the chat history (in the contents array for the role “model” has an old value, there seems to be no way to tell Gemini to prioritize the system_instruction over the chat history. I’ve asked this question to Google Cloud Support and keep getting told they are working on it but no real answers yet. Has anyone experienced this?
Are there any “secret” parameters we can pass to the API call to give priority to the context?
I’ve tried all sorts of instructions and prompting methods, temperature 0 etc. but once an answer is in the chat history, that seems to be used to generate the next answer.
One of the ways I’d approach this is by timestamping the data in the system message and make sure it’s included in model messages as well, as it is time-series data.
e.g. consider a message [04-30-2024 20:10:13] - XYZA at $12.04 by model, then the next API/function call made for the same ticker at 20:10:30 on the same day will be disambiguated with the timestamp and will not “confuse” the model.
Interesting idea. So timestamp the contents array, both the model and user roles? Then in the system_instruction, also add a timestamp? Should I add instruction to use the system_information if the timestamp is later than the contents? I’ll try this tomorrow, but just wondering if you have had success with this approach?
Is there a preferred timestamp format Gemini prefers e.g. [yyyy-mm-dd hh:mm:ss] ?
Yes, you can timestamp the system, model and user messages programmatically in a consistent format or you can ask the model to cite the timestamp from system messages.
AFAIK there isn’t any preferred timestamp for the model.
The model should then be able to infer the latest data. You can explicitly ask it to consider the latest timestamp in case it doesn’t do that on its own.
I didn’t have much luck with this approach. I found that timestamping the model would cause the model to include that timestamp in the answer sometimes (which I can strip out), so I tried just timestamping the user part then both parts and also the context. But Gemini still uses the conversation to work out the answer in favour of the context. Very frustrating!