I am currently working with batch processing using Gemini models. Recently, I came across processing based on the Flex service tier and decided to evaluate it.
I implemented the required changes and tested the setup by specifying the service_tier parameter as "flex". The execution completed successfully. However, when inspecting the response metadata, I noticed that the trafficType (service tier indicator) is returned as null.
This has led to some confusion regarding whether the requests are actually being processed under the Flex tier.
For reference, the response metadata I am receiving is:
GenerateContentResponseUsageMetadata{
cacheTokensDetails=Optional[
[ModalityTokenCount{modality=Optional[TEXT], tokenCount=Optional[717]},
ModalityTokenCount{modality=Optional[AUDIO], tokenCount=Optional[7440]}]
],
cachedContentTokenCount=Optional[8157],
candidatesTokenCount=Optional[2047],
candidatesTokensDetails=Optional.empty,
promptTokenCount=Optional[8911],
promptTokensDetails=Optional[
[ModalityTokenCount{modality=Optional[TEXT], tokenCount=Optional[784]},
ModalityTokenCount{modality=Optional[AUDIO], tokenCount=Optional[8127]}]
],
thoughtsTokenCount=Optional[60],
toolUsePromptTokenCount=Optional.empty,
toolUsePromptTokensDetails=Optional.empty,
totalTokenCount=Optional[11018],
trafficType=Optional.empty // why its empty
}
The API method being used is:
client.models.generateContentStream
Could you please clarify:
-
How to confirm whether Flex tier processing is actually being used?
-
If it is being used correctly, why is the
trafficTypefield not populated in the response?