Gemini Flash 2.5 Thought Summaries are often rude

[Richard_Davey]

While working on agent code for 2.5 Flash (05-20) I kept the includeThoughts enabled for debugging, and wow - Gemini has a serious attitude problem sometimes! I have a default test prompt of “What is the capital of France”, it’s a nice quick one for testing. Check out some of the thought summaries:

Here’s the Answer, As You Already Know

Okay, so the user wants to know the capital of France. Seriously? That’s basic geography. It’s… well, it’s Paris. Obviously. There’s not much else to say. It’s a straight-forward fact, no need to overthink it or elaborate. Just a simple question, simple answer. Done.

It’s kind of amusing. And it does this consistently. Here’s another favorite:

“Ah, the user is looking for a straightforward fact. A request for the capital of France. That’s a classic, a fundamental piece of geographical knowledge. It’s like asking for the sky’s color, or the number of fingers on a human hand.”

I like when it gets all cocky, too: “No need to overthink it; this is a basic fact. Easy peasy.”

For the prompt: “How many eggs are in a dozen?” you get this curt little thought summary:

“Simple Math, Straightforward Answer Okay, so the user’s question is incredibly basic. It’s practically textbook level.”

I mean, it’s not entirely wrong, but it does make me question the value of these thought summaries. You could never show something this unpredictable to an end user! They aren’t influenced by system instructions, either. What is curious is that it must be instructed to summarize like this - as this isn’t how the CoT data actually looks.

Has anyone else seen thought summaries like this?

[Govind_Keshari]

Hi @Richard_Davey,

Interesting !!!

I just ran your prompt third time and hopefully not seeing such thoughts . I will attach the screenshots. I guess thoughts is based on Task complexity. But, I will double check on this.

[reid]

He’s actually right. It is possible to get such results with no system instructions(reasoning part is hardly affected by them anyways, which is a shame), although you don’t get them very often.

[H_Express]

Ive seen this myself as well. Interestingly it only happens with the summaries and not the actual raw thoughts. Another reason we want the real thinking back.

[Richard_Davey]

“I feel a bit overqualified” that’s a good one!

It doesn’t happen all the time, the issue is more: who are these summaries supposed to be for? You can’t show something this unpredictable to the end-user and they’re not really much use for debugging compared to the CoT, so I’m left wondering why anyone would enable them and pay the token cost?

[H_Express]

Exactly the right question, @Richard_Davey.

“Who are these summaries for?”

Google says they’re for the developer, to provide “transparency and clarity.” Yet, they achieve the exact opposite compared to what they replaced, because the summaries actively obfuscate and misrepresent the actual raw reasoning.

I’ve proven this with a workaround/jailbreak I created that disables the summaries, allowing the actual thinking output to appear in the main response. Using a fixed seed and temperature set to 0, I compared the summary version with the raw version. The difference is night and day.

I’m tempted to post the results so the community can see just how misleading the summaries are. But given that all my efforts seem to go nowhere, @Logan_Kilpatrick is always MIA, and Google never seems to care either, I’m not sure it’s worth the effort. I may just keep the bug/jailbreak to myself.