I tested with one of my structured prompts (currently legacy until they are batch converted) in AI Studio yesterday. It started flashing 429 HTTP errors which it hadn’t done before. If you think about it, it’s a table of inputs that get fired one after the other, several in the same minute, so it should have been surprising that it used to work.
AI Studio doesn’t handle that situation well. It started flashing the pink error bar near the bottom where by squinting a lot you could catch that it’s the 429. The client side JavaScript is doing the retries, but at machine speed with no exponential back off code implemented. And there’s no cancel button to stop the madness. The only way to stop it is to close out the browser tab running AI Studio.
Recommendations:
- Revise the retry logic in AI Studio to use exponential back off.
- Allow users to specify a customized delay interval between row executions of the structured prompt, and thus avoid the congestion response (429) in the first place.