429 Issue with a large prompt

I’ve got a weird issue with Gemini 1.5 Pro (seems to be only that) and a large-ish prompt. When I test my prompt in AI Studio, I get a token count of around 170k or so and I get a result just fine.

I then do a quick test with the Node SDK and I can’t get the prompt to run. Calling countTokens returns 185530, well within limits, but when generateContent is run, I get a 429 Too Many Requests. Now, you could think that’s just me using the API too much, but I’m not, and can confirm in the cloud console. I can also immediately run a test with a small prompt and it runs fine.

I created a repro here: ai-testingzone/test_pro_issue at main · cfjedimaster/ai-testingzone · GitHub. Obviously you need your own key. When I run my script, test.mjs, against a small prompt, it works perfectly. When I run it against the larger one (large.txt), I get the 429 error.

Anything come to mind?

Are you using an API Key that has billing enabled or that is on the free tier?

Gemini 1.5 Pro on the free tier is 32000 Tokens Per Minute. Even one attempt with a large context window would exceed that and trigger the 429.

Bam, that must be it. I’m on the free tier. Thank you - so obvious. So basically, no way to come even close to using the full window of Pro on the free tier.

Not with 1.5 Pro, no.
You do have a larger quota with 1.5 Flash, which may be sufficient for testing your app until you switch to Pro for a higher quality testing on the paid tier, but it won’t be enough to use Pro all the time for larger contexts.

In my case, I was doing a test of Flash vs Pro, and figured, the larger the input, the greater the difference in speed. Not the end of the world, just testing, and now I know. Thank you!