I don’t know why but the model keeps dumber in my codebase. It cant even understand simple A to A’ problems in my code or it keeps hallucinating the problem statement then adds random generate on its own instead of fixing what I just needed to fix.
Hi @Myth_Xenodo ,
Could you please share us the following details
- The specific model name used.
- We appreciate it if you attach Colab notebook demonstrating steps to reproduce the hallucination.
- Clarification on whether you tested this across other models and got different results.
Bearing in mind that it isn’t a frontier model, this kind of thing does happen sometimes—though that’s true for almost every model in this tier.
When it happens to me, I personally just tell it not to waste time and switch strategies. In fact, I often ask it to generate 3 or 4 different approaches, evaluate them, and pick the best one. Since it’s so fast, it does this instantly and always ends up solving the issue. I’m really happy with it.