For some reason, I can’t get any of the 2.5 models to generate an answer in chats that are over 960K tokens after a certain point. (I know it’s a lot )
I get this:
Failed to generate content, invalid input. Please adjust your prompt and try again.
If I delete the message (and the error), and rerun this turn for the previous message(s), it sill works fine, but the moment you send another prompt of any kind where it errored out, it fails with that error regardless.
Good evening , please remember if you check your right hand tab which contains the info about controls over the thought length context and contextual window limit . You’ll notice that it’s set just just over 1 Million due to some extent your contextual window has perhaps reached its limits?
I have the same problem.
It started about 14 hours ago. Token Count has not been exceeded.
Everything works in new chats, and it also works on models below 2.5.
Deleting old messages to clear the context does not help. I deleted about 70k tokens, but everything is still broken.
Copying the chat does not help either; the copied version has the same error.
If you wait about an hour and rerun it, the generation will take about 30 seconds, after which it will give an error. If you rerun it immediately after the error, the generation will take only 4 seconds and immediately give an error.
All Tools are turned off. Advanced Settings are default.
Alright, I’ll just describe what I did and what resulted. I wasn’t able to solve the issue or pinpoint the exact cause, but perhaps someone else might find something useful here.
Here’s the error: when making a POST request to GenerateContent, I receive the following response: “The input token count exceeds the maximum number of tokens allowed (1048576).” This is what’s later displayed in the UI as “An internal error has occurred.”
However, the actual token count in my chat is only 800k out of the allowed 1,000,000+.
Next, I used “Get SDK code to chat with Gemini”. After cleaning it from all unnecessary symbols ({}`'][*#, etc.) and executable code, I ended up with a clean Q&A text of the chat.
When I input this entire cleaned text into a new chat, the error reappears, as expected. But I found an exact token boundary for the error: at 741,126 tokens everything works fine, but at 741,127 tokens, it fails.
Interestingly, if I take a book, say “The Count of Monte Cristo,” and paste it entirely into the chat with a simple ctrl+c, ctrl+v, the model easily handles and responds even when the context exceeds 900k tokens.
Moreover, if I fill the initial message with repeated ‘A’ characters, the token limit significantly decreases. I tested only down to around 600 tokens because the browser tab’s memory usage became extreme (up to 5GB), causing severe UI lagging. Reducing the number of characters didn’t substantially alleviate this issue, and the tests became very time-consuming to perform.
Therefore, I still haven’t identified precisely why or how this specific issue arises. The limit of 741,126 tokens only seems to trigger on my specific unique text. With other texts, either no limit or a higher limit applies, as seen with “The Count of Monte Cristo,” where I stopped testing due to fatigue.
I’m not sure if this relates to the number of messages or not. In the chat with the 741,126 token limit, there were about 4 messages from me and roughly 8 from the model, as it responded multiple times to one large message. I chose not to test the scenario of reaching exactly 741,126 tokens in a single initial message, as that again creates severe browser performance issues with processing such large input.
Hi,
May I ask if you are facing this issue with AI studio only or Gemini API as well?
This is only in AI studio for me.
But I haven’t checked it in API usage. I can check it in API for my case tomorrow.
I only tried the AI Studio but if I had to guess, it would most likely happen with the API too as it seems to be an Internal error with the 2.5 models
(It errors out at around 5 seconds after execution)
Hi @denver
Are you facing this issue in English or some other language?
The only chat that I have that has this issue is in English with a bit of French in some messages.
Also do you observe this issue while using document understanding/ video understanding/ image understanding or also on with simple long chats as well?
There was image understanding yes, altough some images that were previously in chat would be gone for no reason eventually (only the message that was associated with it would be still here), I don’t know if that’s intentional or not, but that didn’t prevent me from continuing before I got that error.
Hello,
This issue should not occur with the Gemini API. AI Studio is primarily meant for experimentation to explore different models and features, but it may sometimes struggle with stability.
For more reliable performance, we recommend using the Gemini API.
If you encounter this error while using the Gemini API, please feel free to raise the issue.
