First, i told 1.5 Pro 002 to make a recap of the rules of System Instructions, next I tried to send a token file of 228,068 tokens to make the new model do the entire recap of a fictional tale, next it did so with the story, the characters, and the major deaths. But the next prompt at attempting a Scene and Dialogue, it refuse to obey despite all settings of safety set to ‘None’ What kind of bug is this? And now I’m slapped in the face with a “You’ve reached your rate limit Please try again later.” after trying again over and over again to make it obey. I expected bug fixing in this new version, instead I got more problems to deal with.
That’s about how they had to change context management. Of course, its never going to be the same as early closed gemini 1.5 but then that was completely not realistic to keep. Any idea how many cards it took to hold 1 user 1M context?
Anyways, out of them all since, I actually found it managed the best
If yer talking about the output from a single 200k input, Its your prompting, my friend
200,000 tokens is by no means a small length. If the context length gets too long, in the worst-case scenario, it might completely fail to follow the instructions.
From the Gemini model’s perspective, it’s like briefly glancing at a very detailed and intricate manual and then reading a very long book. (And that’s exactly what it’s like.) Humans, while reading a book, can ‘output’ thoughts about what they need to remember. This means they can remind themselves and avoid forgetting. However, the model can only produce output after processing all the input.
My advice is to place the system instructions after the file tokens. If you include them in the system instruction field and also add them after the file input, you’ll see significantly better results.
