Field `usageMetadata` is breaking code. Is it an intentional, permanent change to the API?

Gemini API
,
1

As of mid-day May 2, the endpoint https://generativelanguage.googleapis.com/v1beta/ appends this object to server responses after the “candidates” array:

  "usageMetadata": {
    "candidatesTokenCount": 27
  }

This post has the bug tag because that means the messages returned from the endpoint no longer conform to the specification for GenerateContentResponse

This change broke application code, for example the go library started showing

proto: (line 34:3): unknown field "usageMetadata"

(This is reported in proto: (line 34:3): unknown field "usageMetadata" · Issue #97 · google/generative-ai-go · GitHub )

Two questions:

  1. Is the new observed behavior a permanent change, and the documentation for GenerateContentResponse will be updated soon?
  2. The specification for Candidate (Candidate  |  Google AI for Developers  |  Google for Developers) includes the member “tokenCount”: integer, which has been missing in the responses from the v1beta endpoint (this has been previously reported and is a known bug). Based on its placement, we are currently assuming that candidatesTokenCount (in usageMetadata) will contain the sum of the tokenCount over the array of Candidate. Since the candidate count is currently fixed at 1, the candidatesTokenCount may be a suitable substitute for the missing tokenCount field. Is that the intention?

Receiving a response from an authoritative source is valuable for developers using the API.

2 Likes
2

After backing out the breaking change, the change was re-applied to the environment on May 9th. The chances are good that it will not be a breaking change this time around. An actual generate content server response now has the form


{
  "candidates": [
    {
      "content": {
        "parts": [
          {
            "text": "Nothing much, just ready to debug some code! Hit me with your snippets and I'll do my best to find any lurking errors.  \n"
          }
        ],
        "role": "model"
      },
      "finishReason": "STOP",
      "index": 0,
      "safetyRatings": [
        {
          "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_HATE_SPEECH",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_HARASSMENT",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
          "probability": "NEGLIGIBLE"
        }
      ]
    }
  ],
  "usageMetadata": {
    "promptTokenCount": 20,
    "candidatesTokenCount": 28,
    "totalTokenCount": 48
  }
}

The server message no longer conforms to the documented specification for GenerateContentResponse in GenerateContentResponse  |  Google AI for Developers  |  Google for Developers which is why this post carries the bug tag.

1 Like
3

The new field has been updated in the documentation: GenerateContentResponse  |  Google AI for Developers  |  Google for Developers and this resolves the bug.

1 Like