Critical Hallucination: Gemini falsely advised on Billing Settings ("Enable AI Credit Overages") causing unexpected charges

Google AI Studio
1

I am reporting a severe safety and hallucination issue regarding the Gemini model.

When I asked Gemini about the “Enable AI Credit Overages” setting, it confidently provided false information. It told me that it acts as a “safe insurance” and “will only use the pooled money if the flat-rate limit is exceeded,” urging me to turn it ON.
Trusting this official Google AI, I enabled it, which resulted in a massive, unexpected API charge due to an inefficient batch process looping in a single turn.

I am currently requesting a refund through Cloud Billing Support (Case #74219190).
However, it is extremely dangerous for the AI to hallucinate and provide false specifications about Google’s own billing/payment systems, directly leading users into financial harm.

I strongly urge the development team to implement stricter guardrails to prevent the AI from giving false advice on billing and account settings.

(Chat URL: https://gemini.google.com/app/29ba03cfdea97ec3 )

2

Hi, Generative AI models (like Google Gemini) can occasionally produce inaccurate responses or hallucinations. Model outputs should always be cross-referenced with official documentations before making significant changes like production pipelines or configs that impact financials. For example, enabling payment settings or running large batch workflows.

Most major AI providers (including Google Gemini) provide this disclaimer on the chat window.

User provided screenshot