Last week I tried something fun and romantic: I wanted to give my boyfriend a special birthday surprise, so I fine-tuned a large-language model with Google Vertex AI using our past chat history. The goal was for the model to know my memories and being able to reply exactly like a digital version of me.
I ran the job for about four hours at night and was confident it’d stay under the free USD $300 credits. I even set a £5 budget alert. The next morning, a few hours after waking up, I opened an email titled “Issue with your Cloud Billing” → £11,550 (€13,600 / $15,500) due.
As a full-time student living on a €425 monthly stipend (after rent I have €212 for everything), this was devastating. I thought the training would cost maybe $150-200. Instead, one short run got me a charge worth years of living expenses.
What went wrong?
- Budget alerts in GCP don’t hard-stop resources. Additionally, the budget warning email didn’t arrive until several hours after I had already discovered the huge charge).
- The UI never warns you when the free credits are exhausted; it simply continues billing against you.
- I was naive about GPU-hour pricing and the volume of parameter-efficient tuning.
What happened next?
- Google Support provided a partial goodwill credit - for which I’m extremely grateful for - reducing the bill to £2,887.72. However, this still accounts for 16 months of my disposable income.
- I shared the story on the platform called Red note; it reached 1.2 million views, 15 k likes, 2 k comments within 48 hours. From the engagement there I could see that many developers fear unexpected cloud bills.
Why I’m telling this story here
- Awareness – Budget alerts ≠ hard limits. There’s no hard limits and you will end up with an astronomical bill using Google Vertex AI, even with spending caps.
- Advice Needed – Have you successfully negotiated further waivers or instalment plans with a cloud vendor? Any DevRel or FinOps folks at Google willing to guide a student?
- Call for Change – Cloud-cost UX still has dangerous blind spots. Students and indie devs deserve real safety rails.
If you have ideas, internal contacts, or volunteer credits that could help, please DM me. I’m happy to share full billing details confidentially.
I still love experimenting with AI and GCP – I just hope this hard lesson can become a catalyst for better cost controls for everyone.
Thank you for reading & amplifying.
Mods: if this post violates rules please let me know – no promotion intended, just genuine help request.
“Modified by moderator”
— Lyahn Wang | Third-year Economics & Management student