Has anyone knows if the amount of tokens Gemini uses to do work is being reduced severily?
It went from top tier model to just garbage. I cant see any other reason why the model 3.1 Pro would make so many mistakes now. I asked it and it said that now it has an “enhaced” quick parsing tool that makes it discover quickly whats relevant to the context and focus on what matters.
If it’s true and it isn’t hallucinating, for me it’s clear that it’s in fact narrowing the context and focusing only on what it’s asked to do, completly skipping the surroundings for context.
So, my guess would be that, trying to make it more efficient, they have reduced the amount of token it ingests before the output, therefore making it prone to halucinations.
Has anyone else noticed that?
Hi there,
Typically, if an AI model runs out of context, it will either begin compressing the conversation or stop working. They do not typically continue to function once they run out of context. This is because they cannot. Their memory would be too full to even appear as though they are still working.
Has the agent displayed an error stating that you need to begin a new chat? In Antigravity, that is usually the clue that your agent has run out of context and can no longer handle more of the conversation.
There could be many reasons why you are seeing degraded performance. Over the last few years, with various AI models from different companies, I have noticed that users are often very happy at the beginning of their projects and then tend to feel like the AI is getting “dumber” the more they use it over time. I personally believe this is related to the way they are organizing their project structure and possibly becoming too comfortable with, or fatigued by, writing prompts.
Do you typically keep a GEMINI.md file in your codebase? You can initialize one by running /init. The agent will scan your codebase and create a file that it can later use to help keep track of different tasks and important information about your project.
Additionally, when the agent creates a .md file with information about a task it completed, do you typically keep those files? I put them in a folder called docs and point the agent to it when I know that information will be useful.
Important: I highly suggest prompting the agent with only one task at a time, or doing so until you move on to a different part of your codebase. Sometimes I forget to do this, and the agent gets confused. Here is why:
When you send a message to the agent and it sends one back, then you reply again, the agent is receiving the entire conversation every time you send a message. After 20 to 30 messages, if the chat starts taking different turns and moving in different directions, what the agent read at the very beginning may no longer be relevant or true 20 messages later. This is why it is best to group your conversations by task or by area of your codebase.
I hope this helps.
Regards,
Ray
Hi there, @TechRay.
I see that my wording creates confusion, so I’ll rephrase it. I didn’t mean context as per the tokens cached at it’s memory. I’m saying context = awareness of the full picture. Before, through the thinking process, you could see that it would go in depth into the file to undestand it. Now, it repeats a call for (Analyze request and see which tools to use) and then starts implementing, spending much much less thinking on the task before executing.
I’m going to give a real example:
Before, while in deep mode I would ask him ,for example, to add features on a main.py of a project, and it would go through the code and understand how the different parts of the code would interact, thus anticipating that certain changes would have a potential disruptive effect in other features.
Recently, I had him go over a file that was of a considerate size code-wise. I had asked him to help me debug an issue I was having with a function.. He went through it a bit and then simply decided TO ERASE THE WHOLE FILE CODE AND REWRITE JUST A FEW LINES SO THAT THIS FUNCTION WOULD WORK. I checked what was his thought process later and I saw that it tried to correct the bug three times but was having an issue with a variable. After the third time it decided that it would be more efficient to delete the code and start fresh to ensure that the code would be flawless.
Luckily there is the review to accept or reject his changes, but I was astonished that it decided to do that.
So, basically this is what I meant. I asked him later what was happening and it said that there was this quick parsing feature that made him more efficient so he would focus on the task and not waste tokens going through unnecessary material.
Another example. Just look at this. I clearly stated that columns from D to H were merged and it hallucinated out of nowhere that rows D2 to D5 were merged (which aren’t). And this was a 5 lines prompt. And this Gemini PRO not even Flash. This is what I meant
@TechRay
Before all these changes, I had never seen anything like this kind of error on Gemini.
