Hello, has anyone else faced the issue when trying to input a large pdf (500k+ tokens) into the model, it used to work flawlessly before but since last month it would always put out a error message saying “unable to count token” or something along the line of that. When will they fix this issue?
No, but which model are you using? (Exact model with version)
Any model really, be it the flash or the pro model. Even happens with the experimental models too! I am shocked that its not the same for everyone, have you checked? Is it working for you when you upload a big text/pdf file to the model?
1 Like
I tried images and even videos. You’d surpass 8K context pretty quick. What language and API do you use?
I don’t use the api but rather just the aistudio site since i just use it to explore literary works if you wanna call it that, it used to work flawlessly before but since last month it just started throwing error whenever i try to add big pdf files, like even something thats like 270,000 tokens at most fails even tho it used to work before.
1 Like
Hello! Have you checked that your Google Drive is not full or about to be full? I had the problem that it wouldn’t let me upload large files, but it was because my Drive was almost full and the document wouldn’t fit because of the weight.
I’ve found that this usually happens when you try to upload files types that aren’t supported. In your case though, I’m not sure what’s happening.
My google drive on my main account is 80% full but i know that’s not the problem because i have another fresh google drive account and the issue persists even on that account too. So yeah that’s not the problem but thanks for suggesting that since it helps troubleshooting the issue.
Unless they stopped supporting pdf files then i don’t think that would be the case, because i know that these exact large pdf files was working properly a month back and just stopped working suddenly, if you have the time i want you to download some large pdf file and try it for yourself to find the problem, try harry potter books since the pdf version of those books are publicly available. It seems like only i am facing the issue here so i wanna confirm that
There’s another limit that was introduced (wasn’t there before); PDF must be 300 pages or less. Straight from the product manager - "Failed to count tokens for this attachment" Error - #11 by Logan_Kilpatrick
Hope that helps.
2 Likes