I’m using Vertex AI API with flash 2.5 model version 4-17 and running into an issue where I get the following error:
{‘code’: 400, ‘message’: ‘Request contains an invalid argument.’, ‘status’: ‘INVALID_ARGUMENT’}
I’ve traced it back to a list of enums in my structured output. When I reduce the list to just 3 options, everything works fine, but with ~100 options, it fails. The input size is around 6000 tokens. I also tried switching from enums to literals, but the error still occurs.
Has anyone encountered this before or found a workaround?
The error you’re encountering—400 Invalid Argument—is likely due to the number of enum values in your response schema.. Community feedback suggests that schemas with around 100 enum values can lead to this error however Gemini API doesn’t explicitly document a max number of enum values.
Try to reduce the number of enum values or use literal values instead of enums
I am also experiencing the same issue. I am trying to do a multi-class classification task where I am tying to classify certain texts into any one of the given classes. I could have upto 1500 classes in the Enum list and as you mentioned I also get the same error and model does not strictly adhere to picking classes from given Enum list even for say a 100 classes in the list.
However, when I tried using FunctionDeclaration and FunctionCallingConfig class and provided for a structured output via function call object. I observed that the model mostly classifies into the classes in the given Enum list. It is still not perfect but hallucinations or model coming up with its own class labels has reduced a lot. Refer: Function calling reference | Generative AI on Vertex AI | Google Cloud. But when I tried to do the same via Batch Prediction job the model goes crazy and does not follow the function call.
