Function call string property is double escaped

Gemini API
,
1

Why characters like " or ' or \n are getting double escaped in value of string properties of function call args?

I’m getting such response from Gemini Flash:

'\\"\\"\\"\\n#include <iostream>\\n\\nint main() {\\n  std::cout << \\"Hello, world!\\" << std::endl;\\n  return 0;\\n}\\n\\"\\"\\"'

Gemini Pro answer is:

'// Your First C++ Program\\n\\n#include <iostream>\\n\\nint main() {\\n  std::cout << "Hello World!";\\n  return 0;\\n}'

(so slightly better, because it is not wrapped with """ but still the \n is double escaped)

while I would expect:

'"""\n#include <iostream>\n\nint main() {\n  std::cout << "Hello, world!" << std::endl;\n  return 0;\n}n"""'

The workaround is to do such find/replace: .replace(/\\n/g, '\n').replace(/\\'/g, "'") but unfortunately it is flaky. Sometimes a \\n is valid (inside string literal).

This problem can be reproduced with this node.js script:

// Derived from: https://github.com/GoogleCloudPlatform/nodejs-docs-samples/blob/main/generative-ai/snippets/function-calling/functionCallingBasic.js

const { VertexAI, FunctionDeclarationSchemaType } = require('@google-cloud/vertexai');

const functionDeclarations = [
  {
    function_declarations: [
      {
        name: 'saveSourceCode',
        description: 'save source code to a file',
        parameters: {
          type: FunctionDeclarationSchemaType.OBJECT,
          properties: {
            path: { type: FunctionDeclarationSchemaType.STRING },
            content: {
              type: FunctionDeclarationSchemaType.STRING,
            },
          },
          required: ['path', 'content'],
        },
      },
    ],
  },
];

/**
 * TODO(developer): Update these variables before running the sample.
 */
async function functionCallingBasic(
  projectId = process.env.GOOGLE_CLOUD_PROJECT,
  location = 'us-central1',
  model = 'gemini-1.5-flash-001',
) {
  // Initialize Vertex with your Cloud project and location
  const vertexAI = new VertexAI({ project: projectId, location: location });

  // Instantiate the model
  const generativeModel = vertexAI.preview.getGenerativeModel({
    model: model,
  });

  const request = {
    contents: [{ role: 'user', parts: [{ text: 'Generate a c++ hello world' }] }],
    tools: functionDeclarations,
    toolConfig: {
      functionCallingConfig: {
        allowedFunctionNames: ['saveSourceCode'],
        mode: 'ANY',
      },
    },
  };
  const result = await generativeModel.generateContent(request);
  console.log(result.response.candidates[0].content.parts[0].functionCall);
}
// [END generativeaionvertexai_function_calling_basic]

functionCallingBasic(...process.argv.slice(2)).catch((err) => {
  console.error(err.message);
  process.exitCode = 1;
});

I will appreciate help with this :slight_smile:

ps. This problem does not happen on other ai service providers as I checked

2 Likes
2

The problem becomes more apparent for such prompt:
Generate a node.js hello world that takes user input, use single aposthrophes for strings:

image
image2182×226 23.1 KB

3

Interestingly, I’m getting correct result when I’m using structured output instead of function calling:

image
image1114×346 27.9 KB

1 Like
4

I’m running into the exact same issue. @Grzegorz_Tanczyk would you be so kind as to share how you’ve worked around it using structured output?

5

I don’t consider this as a proper workaround, as I need to use function calling in my use case. I posted this example to have a reference point for this bug report.

I used the standard configuration for structured output:

    generationConfig: {
      responseMimeType: 'application/json',
      responseSchema: {
        type: FunctionDeclarationSchemaType.OBJECT,
        properties: {
          path: { type: FunctionDeclarationSchemaType.STRING },
          content: {
            type: FunctionDeclarationSchemaType.STRING,
            description: 'source code',
          },
        },
        required: ['path', 'content'],
      },
    },
1 Like
6

Thank you. My hunch is that Gemini is trying hard to output markdown but I don’t have any real evidence. I’ve tried prompting it to not apply any formatting, etc but it still insists on double escaping things and adding \n sequences all over the place.

1 Like
7

Same here :confused: Otherwise works promisingly good compared to other models

2 Likes
8

I am with you that as it stands this is unfixable, can not be worked around, short of using alternative tool implementations.

You can completely break it by defining an echo tool and just asking it to echo something like

puts \
puts \n

At that point the tool implementation gets so confused that it does not even output the first newline:

data: {"candidates": [{"content": {"parts": [{"functionCall": {"name": "echo","args": {"string": "puts \\\\ puts \\\\n"}}}],"role": "model"},"index": 0,"safetyRatings": [{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"}]}],"usageMetadata": {"promptTokenCount": 81,"candidatesTokenCount": 19,"totalTokenCount": 100},"modelVersion": "gemini-exp-1121"}

Basically makes tools unusable for source code sadly.

1 Like
9

Here is the unescape function I’m using: genaicode/src/ai-service/unescape-function-call.ts at master · gtanczyk/genaicode · GitHub

it helps, does not cover 100% of cases unfortunately (some are as @Sam_Saffron described: unfixable), but unblocks usage of Gemini in the source code use case. I have :crossed_fingers: for a proper resolution on API side eventually.

2 Likes
10

I raised this here:

Hopefully the library writers can escalate internally :pray:

2 Likes
11

Great news, this is now fixed !

2 Likes