The situation is that, I do not know when it started, probably this year, all the models are very slow to reply after many conversations, making the web page unresponsive for tens of seconds or even a minute to reply. And when I input the content, I can obviously feel that there is a delay in the input, especially if the auto save is turned on, the reply content of the ai will be blocked for tens of seconds after changing a few words.
I tested it and it wasn’t a web page or device issue. I used two computers and two phones. It was the same with edge, Firefox, bing, etc.
I guess it should not be the problem of tokens, my longest is only 30 or 40 thousand tokens, far from reaching the upper limit of more than one million tokens. And I found that token does not seem to affect the lag, the main factor should be the number of dialogue rounds. For a conversation with a lot of replies, and a conversation with a few replies, there was a lag of about a second after about 30 or 50 conversations.
At present, all I can do is to start a new dialogue after too many dialogue rounds cause serious delay, and then integrate all the content of the original dialogue and copy it to the new dialogue. This only takes one dialogue round, and the corresponding speed is very significant. However, this is very cumbersome and affects the user experience.
Hopefully Google will fix this issue soon. gemini is by far the best big ai model I’ve ever used and the best fit for my use case, but I’m suffering from this lag problem. Hopefully Google will fix this issue soon.
7 Likes