Hi folks,
I have this problem where I have to send a lot of fast requests to flash in a row, in average they take between 1.5 to 2s, but sometimes one of these requests is going to either take an unusually long time (think 2mn for the same context length) or crash with something like below
{ kind: Request, url: “https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent”, source: hyper_util::client::legacy::Error(SendRequest, hyper::Error(Io, Custom { kind: BrokenPipe, error: “stream closed because of a broken pipe” })) }
Any idea why taht happens?
This completely breaks my product and we could spend a LOT on Flash 2.0 if it actually worked reliably and didn’t crash every 5mn