The model, despite instructions during code creation, exceptionally violates user guidelines and ruins code.
Even when forbidden from operating on specific code sections, the model will still completely change the forbidden sections when providing code.
Ignoring user guidelines and system instructions. Example: the model is supposed to fix a few things in my chatbot, but gemini focuses mostly not on the task but on changing the model used in the chatbot and rearranging parameters that the user had established.
Completely ignoring forbidden things. Example. I forbid using libraries X and Y, and instead suggest Z, the model will ignore this anyway and mindlessly use X or Y.
As a joke, I made system instructions with 100,000 tokens, where there was just one sentence copied hundreds of times saying ‘You are forbidden from X and Y’, and the model ignored it during operation anyway!
I feel that something went wrong. The model cares even less about system instructions and user guidelines than previous versions.
This is not directly code related, but it is similar behavior.
Gemini 2.5 is driving me absolutely batty with its footnote behavior. I have a page and a half of instructions. Two chunks:
How do footnotes
How to verify them
It does neither consistently. Of course, it does do some things spectacularly well… like the research and writing I’m having it do. I still have to edit, of course… I just wish it would follow the basics.
Overall quality in architect-mode is good, but the model has severe issues with editing larger files and I thought moving away from Claude 3.7 Thinking, because issue with context -window limits.
I’d not recommend to use 2.5 Pro exclusively and combine it with Claude. Also it gets quite expensive, especially if it has loop-issues/loosing context Just spent €90, as it kept wrong editing and i didn’t pay attention (expensive lunch-break )
In overall, I really like 2.5 Pro, because the 1m token window are what I sometimes need, but after the poor performance on API side (yes, also paid tier has over the day extreme issues/timeouts) I need to move on and build my own context-memory with better performing models.
I’m noticing very strange behavior. Now ADK has been released. I thought great, I’ll be able to create amazing workflows and agent networks using 2.5. The result? I’ve never been so angry at Gemini. Instead of using ADK properly (I give it formatted information from the website and GitHub, provide examples), the 2.5 model still mixes up libraries, randomly throws in generativeai, inserts something like gemini-pro-vision. And I almost forgot to mention it makes 5 comments about ‘temperature: 0.7 #to add creativity this is very important very very’. I feel like the model has completely started ignoring system instructions, user input, tasks, and information, and instead works like one big random machine designed to irritate users.
Just spent €90, as it kept wrong editing and i didn’t pay attention (expensive lunch-break