In other words: “No, we’re not going to return raw CoT. Deal with it.”
5 Likes
I think may be you are thinking that people are shallow thinking, they do not need complex, long chain of thought which are in-depth, clear, and is providing strong rationale. You are thinking that people might find it hard to understand and do not want it. People are surface creatures which might love shallow, surface, pleasant replies.
But no, people actually really respect and love in-depth, clear, long chain of though which they find inspiring. They want to see extremely high quality chain of thought in the end.
9 Likes
The summaries will never, ever be better. Just admit it’s not coming back already, even though literally EVERYONE agrees it’s worse and wants it back.
7 Likes
give us a concrete reason as to WHY you cannot give us even just a toggle for summaries AND raw thoughts if we so wish.
summaries IN THEIR CURRENT FORM are useless for all intents and purposes. if corporate has decided we cannot have raw thoughts, just come up front and tell us we can’t have them.
7 Likes
I recently see that shorter COT might lead to more accurate results. Maybe that’s the reason?
On the other hand, high quality, sharp, clear, comprehensive long chain of thoughts in many cases are really valuable, even more valuable than answers.
Maybe let people choose if they want short Chain of Thought but accurate answers, and long chain of thoughts which really have high quality thinking process?
2 Likes
Maybe this is the reason?
This is a completely unacceptable, and frankly, unprofessional response. Shame on you for posting it. This is not “getting better” or “optimizing”, this is taking steps back. This is actively making it harder for those of us interested in prompting and devising custom instructions to actually learn how to do those things. Summaries offer no useful information whatsoever. I will absolutely not be renewing my Gemini subscription until you people finally admit you made a mistake and roll it back. You claim that we’re “valued collaborators”. Then why aren’t you actively assuring us that you’re going to make changes or give us the ability to choose? Google has been one of the world’s leading tech giants for over twenty years, and time after time, we’ve only ever seen them shoot down their interesting, innovative tech and remove genuinely useful features for the sake of appeasing a small number of complaints (or worse yet, shareholders… ugh.).
We’re all human. Humans love choice. Give us choices. Give us the ability to toggle summaries on or off. Do right by your customers and they’ll come flooding back.
9 Likes
I really want to emphatize that the COT was very useful and sometimes it was more insightful than the actual answer.
For example , it could solve the infamous issue of AIs being hyper validating and sycophantic by catching your own mistake within the COT.
The reply might soften its answers, but reading the COT, you could realize your own argumentative flaws.
The ability to see the reasoning helps the users understand the AI’s thought process, facilitating debugging and identifying potential biases or hallucinations. This builds user confidence and empowers them to refine their interactions, as they learn how the AI is “thinking”.
Sure, the COT may be too complex for many users, but that’s why its something that should be a option.
You want a visible COT or not? The user decides.
Understanding the AI’s “thoughts” (even if it’s a simulation of thought) demystifies the technology. Users can learn to craft better prompts and interpret outputs more effectively by seeing what kind of reasoning leads to good or bad results.
Basically, the ideal would be having both: Visible full thinking process, summarization of thinking process and simply the “Don’t show thinking” option. All of them as available options
7 Likes
And yet you are getting worse… Raw thoughts were useful and insightful. Summaries are a waste of space.
2 Likes
If you’re going to never return raw thoughts again, please just say that rigidly and directly, not in PRspeak, so I can get it over with and go local. Transparency is absolutely vital here.
6 Likes
The Chain of Thought is the same as it was, just hidden and we only see summarized version of it (the model uses the full, hidden version).
Hello,
I am truly glad you responded. It would be nice to know why this specific change (degrading summaries) happened so we can understand whether we will see (and what other) such degradations of experience in future.
3 Likes
Lots of good stuff in this thread, catching up on the comments after Google IO craziness.
A few reactions, comments, and clarifications:
- I hear that you all want raw thoughts, the value is clear, there are use cases that require them, and seems reasonable to want them in the API as well
- Why be excited for summaries? The raw thoughts have been disabled in the API for a while now, so summaries was a step closer to allowing devs to programmatically at scale see what the model is doing in reasoning steps. I am still excited about this because there are certainly use cases that will benefit from it (but as folks have pointed out, clearly not all use cases)
- There were a few comments on Gemini 2.5 Pro 05 06 being a regression for certain tasks, we have seen lots of reports of these and are working on some improvements!
- There were some comments on model quality regressions with thinking summaries. To be clear, at the API level, for the last few months, we have not had raw thoughts in the API. The summary is purely cosmetic in that it’s returned after the fact and does not impact model performance in any way, the model still reasons with full thoughts for all queries
- On the Gemini app side, I do personally think this is a real feature improvement. Gemini app is being built as a universal assistant. The % of people who will or do read thought thoughts in the Gemini app is very small, the better product experience on the margin is not showing them (and instead showing summaries, especially as models do more and more in a single pass)
- On the AI Studio side, I hear you all on wanting to have raw thoughts back, it is something we can explore. The dev use case makes perfect sense here and is the reason we left the raw thoughts in AI Studio to begin with when we removed it from the API
- In the short term, we will keep working on ways to make summaries better and see how to expose more
- In the long term, as models do more in the reasoning steps (tool use and otherwise), I can easily imagine that raw thoughts becomes a critical requirement of all AI systems given the increasingly complexity and need for observability + tracing
- For folks using thoughts for function calling / tool calling, we just added Thought signatures which lets you pass an encrypted thought representation in multi-turn conversations that require tool calls: Gemini thinking | Gemini API | Google AI for Developers
- Thought summaries in the API are experimental, specifically so we can get feedback and influence the roadmap, so thank you to those who shared cases where they don’t work well for specific use cases!
I hope this helps!
6 Likes
First, thank you very much for being so honest and direct about this. This is really deeply appreciated, especially the addressing of these points one by one. Better late than never by a long shot. Your candidness about 0506’s backfiring is refreshing too.
Second, again - what matters is choice. Frankly, if you had given us summaries as an option, we’d all be happily giving feedback and trying to improve them. Summaries are definitely an alpha product right now, though they could certainly be much worse. I think the Gemini app, too, should have the option of summaries or raw thoughts. Bury it in a menu, sure, but removing a feature is almost never an improvement.
Third - the element of programmatically analyzing behavior with summaries is clever, and the issue here was miscommunication. If these had been an option, aimed at agentic workflows and easily working with big chunks of work, then I think people would be very happy with it.
This all comes down to one core concept: don’t remove features. Anything that’s lost, people are probably using. Thanks again for the response, and I’m hoping to see raw thoughts come back again someday soon.
P.S. that I forgot to add: can’t wait to see the mojo come back with the next 2.5 Pro, 0325 was an absolutely stunning creative writing tool and I’m really excited to see how the final product comes out.
14 Likes
I appreciate you finally coming here and voicing some of your thoughts, but I want to be absolutely, 100% clear: summarized thoughts do not benefit those of us who are trying to learn prompting and custom instruction writing. We can’t learn from our mistakes, optimize our setups, and combat hallucinations from the AI if we can’t see what it’s truly thinking. All the summaries do is repeat the same variations of “Okay, now I’m thinking about/beginning to understand/zeroing in/figuring out the user’s request for [thing]” for several minutes and then generate a response. Half the time, it’s the exact same summary paragraph repeated, verbatim, 2-3 times in a row. That doesn’t tell me anything. That doesn’t let me learn. That doesn’t help me use Gemini to the fullest.
Doesn’t matter if it’s in AI Studio, the API, or the main web app. We need the option for raw thinking back on every platform. It needs to be available for every user, regardless of their subscription level. This is non-negotiable. We. Need. Choice. You do that, and I’ll subscribe to Gemini for the rest of my life. Hell, I’ll even get the $250 one. Just make this happen.
10 Likes
honestly all we need is raw thoughts in studio and the api
if they want the app to have summaries, so be it. i’m quite content and happy to have thoughts back on studio and the api at the very least.
we should take what we can get
3 Likes
Thank you for engaging with the community, I appreciate the response but I still wonder - why weren’t raw thoughts just left as a toggleable option from the get-go? I feel like that could have avoided a lot of grievances. I understand the feature being removed from the Gemini app for casual users but this thread primarily takes issue with its removal on AI studio, it’s also my impression that AI studio is the platform meant for developers and power users - so why remove it there, when it was already left in specifically for these users? In your words, “The dev use case makes perfect sense here and is the reason we left the raw thoughts in AI Studio to begin with.” Why remove it then, and why can’t it be added back?
Summaries in their current form are simply nowhere near good enough to do any real troubleshooting with - by all means keep improving them, I think people are more than willing to give feedback, but you’re still handicapping devs and power users by not giving the option to access raw cot in the meantime.
9 Likes
Thank you for your attention on this.
Please take a look at what 05/06 answered for a hard-sifi story, a story involve in having a barren realistic planet without any alien life for human colony. a physics/realism grounded word, 05/06 pro even fails to understand what the story mean by giving answer of floating stone, flying jellyfish, flying sky-whale for such a planet what astronomer see! That is ridiculous enough even a Children are not going to answer this. let’s say this story is similar to liucixin’s hard sifi, I am heavily doubt its reasoning capability are capable for task IN and Outside of creative writing.
5 Likes
Thank you. Yes, it’s definitively something necessary for us. It’s a great help to catch if our prompt was flawed. Sincerity Mode, specifically, is a great help. For the short and long term, I understand the intentions.
I definitely think that the Thinking Summaries aren’t inherently bad. The thoughts will absolutely be necessary to see when working with even better and stronger LLMs, as they help users to understand the logic used by the AI (and their own prompts, in the case of System Prompts).
But thanks for listening. I hope that, by the end, we’re all satisfied. Ultimately, Google is now the lead in the AI race, and having a user-friendly Dev is the best outcome.
glad to see the update focused on lowering expectations, consistency is important
4 Likes