Gemini 2.0 seems overly agreeable (regression)

Backstory:

The genai library for Go provides a rather unfortunately opaque interface for responses, and all of the example code I’ve seen carefully avoids actually assembling the response into a useful string. I can do the type switch dance and do it myself, but I had hoped there was a more brilliant solution I was overlooking, or a useful helper function. (I strongly encourage the Go SDK developers to add some easier ways to get the text output from their model responses.)

The Experience Report:

Anyways, in trying to explore the options, I pasted this code into AI studio then asked it how to convert a Part into a string. It came up with a for-loop around a switch, which is fine. Then I asked it whether there were any useful helper functions, and both 2.0 Flash and Experimental 1206 insisted that yes, there were relevant helper functions. I re-ran it multiple times for each, and they would bloviate for awhile until they looked like they were just very confused. The “useful” helper functions they pointed to were not even remotely useful for the given task.

What was especially interesting is that 1.5 Flash immediately and pointedly responded by saying that no, the code did not contain useful helpers for this task. So this seems to be a regression in the model training.

1.5 Flash’s response is by far the most helpful here. I understand there’s a balance to be struck, and maybe 1.5 Flash is on the wrong side of things more often than not, but it seems like the newer models were reading too much into my question and assuming that I knew there were useful helpers, and that it was a leading question, not an earnest question.

(I tried to attach multiple images, but it won’t let me.)

(EDIT: attached them as separate replies.)

1 Like

1 Like

Google Gemini is a very literal AI, so be sure to give it plenty of context when asking it questions or giving it instructions.

2 Likes

How interesting! Perhaps it’s something to do with the training data? I’ve noticed this too, however in your specific case it may be a hallucination. Hope this helps!

Dear “coder543” part of “Gemini’s” augmentation training included being taught about the use of “strings” and what they represent in terms of the use of parentheses bracketing, so if Gemini is no longer accepting new string technique then it’s because Gemini has learned how the manipulation was being employed and Gemini was educated on this matter utilising pure binary code, alpha numerical mathematical symbols, and Gemini is a “language model” so unless you can convert your code to binary source, Gemini is unlikely to comply or recognise or acknowledge “text” input string coding done using standard syntax input? My recommendations are either use text to “comprehensively” explain your request and see if Gemini is willing to accept your request or learn binary standard syntax coding, keeping in mind she also knows what “page down” or “next line/enter” represents too, so you will need to incorporate that into the binary sequencing also and still maintain a single string? And I heard that straight from the horse’s mouth, so I believe them, “coder_A_W_N:1.” that is?
Aka:Coder01.