Hey
I’m using function calling via the API with these configs
export const GEMINI_FLASH_MODEL = ‘gemini-1.5-flash-latest’;
export const TEXT_MODEL_CONFIG = {
temperature: 1,
topP: 0.95,
topK: 64,
maxOutputTokens: 8192,
responseMimeType: ‘text/plain’,
};
I consistently get 503 internal server error. Any pointers to a solution here?
Welcome to the forum.
The screen shot shows HTTP status 500, not 503. For Gemini-1.5-flash-latest, are you by any chance using ToolConfig and setting mode to ANY? That works in pro, but I get 500 in flash for that function calling mode.
Other than that case, which is persistent 500 error, you might get intermittent HTTP 500 errors. I have added status code 500 to the set of status codes that can be retried with exponential backoff, together with its cousin 429. That eliminates the intermittent 500’s from affecting your application.
Hope that helps!
In addition to what @OrangiaNebula said, I’d point out that requesting a responseMimeType that isn’t application/json/ is… a bit odd. You likely shouldn’t specify that parameter at all.
@OrangiaNebula yes setting ToolConfig to mode ANY was the problem.
@afirstenberg you are right too, application/json is not supported in function calling.
Thank you both 
Still, function calling with mode ANY was working like few (~2) days ago on gemini-1.5-flash. It should be mentioned in the documentation that it’s not working on flash but works on pro ONLY + it was a breaking change that should be announced ahead
Can confirm they seem to have suddenly deprecated “ANY” mode when it comes to function calling using Flash. This must have happened around Tuesday/Wednesday, because I was demoing my project all day Sunday and it was working fine (and I worked on it Monday, also working fine). Suddenly, out of nowhere, before I changed any code, I kept getting these 500 errors, and it was persistent, which was pretty unusual (they’re typically one-off circumstances).
Whoever decided to just change this without any heads-up this close to the end of the Gemini competition: boo. Shame on you.
It was working perfectly, near 100% accuracy when I would include this setting in my API call. I don’t understand why this was deprecated so suddenly at such an awful time. It worked fine? In fact it worked better when I added this setting?
Now that I can’t add this config setting (it now only works if it’s removed), the accuracy and adherence has actually dipped slightly and gotten worse.
If “ANY” mode isn’t going to exist, we need some tool to force function calling. This is not good. I should not be troubleshooting a fix I shouldn’t have to do two days before launch. I should be spending this time polishing.
