# "Unable to count token" issue

**URL:** <https://discuss.ai.google.dev/t/unable-to-count-token-issue/39308>\
**Category:** Google AI Studio\
**Tags:** issues, model\_optimization\
**Created:** [September 14, 2024, 9:52pm UTC](https://discuss.ai.google.dev/t/unable-to-count-token-issue/39308 "2024-09-14T21:52:05Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lemonny](https://avatars.discourse-cdn.com/v4/letter/l/f0a364/32.png) [@Lemonny](https://discuss.ai.google.dev/u/Lemonny)\
**Post date:** [September 14, 2024, 9:52pm UTC](https://discuss.ai.google.dev/t/unable-to-count-token-issue/39308/1 "2024-09-14T21:52:05Z")

</div>

Hello, has anyone else faced the issue when trying to input a large pdf (500k+ tokens) into the model, it used to work flawlessly before but since last month it would always put out a error message saying “unable to count token” or something along the line of that. When will they fix this issue?

---

<div class="post-metadata">

**Author:** ![tocsa](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/tocsa/32/21640_2.png) [@tocsa](https://discuss.ai.google.dev/u/tocsa)\
**Post date:** [September 14, 2024, 11:01pm UTC](https://discuss.ai.google.dev/t/unable-to-count-token-issue/39308/2 "2024-09-14T23:01:25Z")

</div>

No, but which model are you using? (Exact model with version)

---

<div class="post-metadata">

**Author:** ![Lemonny](https://avatars.discourse-cdn.com/v4/letter/l/f0a364/32.png) [@Lemonny](https://discuss.ai.google.dev/u/Lemonny)\
**Post date:** [September 14, 2024, 11:59pm UTC](https://discuss.ai.google.dev/t/unable-to-count-token-issue/39308/3 "2024-09-14T23:59:24Z")

</div>

Any model really, be it the flash or the pro model. Even happens with the experimental models too! I am shocked that its not the same for everyone, have you checked? Is it working for you when you upload a big text/pdf file to the model?

---

<div class="post-metadata">

**Author:** ![tocsa](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/tocsa/32/21640_2.png) [@tocsa](https://discuss.ai.google.dev/u/tocsa)\
**Post date:** [September 15, 2024, 1:00am UTC](https://discuss.ai.google.dev/t/unable-to-count-token-issue/39308/4 "2024-09-15T01:00:22Z")

</div>

I tried images and even videos. You’d surpass 8K context pretty quick. What language and API do you use?

---

<div class="post-metadata">

**Author:** ![Lemonny](https://avatars.discourse-cdn.com/v4/letter/l/f0a364/32.png) [@Lemonny](https://discuss.ai.google.dev/u/Lemonny)\
**Post date:** [September 15, 2024, 1:49am UTC](https://discuss.ai.google.dev/t/unable-to-count-token-issue/39308/5 "2024-09-15T01:49:16Z")

</div>

I don’t use the api but rather just the aistudio site since i just use it to explore literary works if you wanna call it that, it used to work flawlessly before but since last month it just started throwing error whenever i try to add big pdf files, like even something thats like 270,000 tokens at most fails even tho it used to work before.

---

<div class="post-metadata">

**Author:** ![Tomas\_Dulanto](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/tomas_dulanto/32/24046_2.png) [@Tomas\_Dulanto](https://discuss.ai.google.dev/u/Tomas_Dulanto)\
**Post date:** [September 16, 2024, 6:40am UTC](https://discuss.ai.google.dev/t/unable-to-count-token-issue/39308/7 "2024-09-16T06:40:22Z")

</div>

Hello! Have you checked that your Google Drive is not full or about to be full? I had the problem that it wouldn’t let me upload large files, but it was because my Drive was almost full and the document wouldn’t fit because of the weight.

---

<div class="post-metadata">

**Author:** ![Andrew\_Mazanec](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/andrew_mazanec/32/814_2.png) [@Andrew\_Mazanec](https://discuss.ai.google.dev/u/Andrew_Mazanec)\
**Post date:** [September 17, 2024, 3:57am UTC](https://discuss.ai.google.dev/t/unable-to-count-token-issue/39308/9 "2024-09-17T03:57:47Z")

</div>

I’ve found that this usually happens when you try to upload files types that aren’t supported. In your case though, I’m not sure what’s happening.

---

<div class="post-metadata">

**Author:** ![Lemonny](https://avatars.discourse-cdn.com/v4/letter/l/f0a364/32.png) [@Lemonny](https://discuss.ai.google.dev/u/Lemonny)\
**Post date:** [September 17, 2024, 1:02pm UTC](https://discuss.ai.google.dev/t/unable-to-count-token-issue/39308/11 "2024-09-17T13:02:07Z")

</div>

My google drive on my main account is 80% full but i know that’s not the problem because i have another fresh google drive account and the issue persists even on that account too. So yeah that’s not the problem but thanks for suggesting that since it helps troubleshooting the issue.

---

<div class="post-metadata">

**Author:** ![Lemonny](https://avatars.discourse-cdn.com/v4/letter/l/f0a364/32.png) [@Lemonny](https://discuss.ai.google.dev/u/Lemonny)\
**Post date:** [September 17, 2024, 1:08pm UTC](https://discuss.ai.google.dev/t/unable-to-count-token-issue/39308/12 "2024-09-17T13:08:52Z")

</div>

Unless they stopped supporting pdf files then i don’t think that would be the case, because i know that these exact large pdf files was working properly a month back and just stopped working suddenly, if you have the time i want you to download some large pdf file and try it for yourself to find the problem, try harry potter books since the pdf version of those books are publicly available. It seems like only i am facing the issue here so i wanna confirm that

---

<div class="post-metadata">

**Author:** ![OrangiaNebula](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/orangianebula/32/480_2.png) [@OrangiaNebula](https://discuss.ai.google.dev/u/OrangiaNebula)\
**Post date:** [October 8, 2024, 5:08pm UTC](https://discuss.ai.google.dev/t/unable-to-count-token-issue/39308/14 "2024-10-08T17:08:07Z")

</div>

There’s another limit that was introduced (wasn’t there before); PDF must be 300 pages or less. Straight from the product manager - ["Failed to count tokens for this attachment" Error - #11 by Logan\_Kilpatrick](https://discuss.ai.google.dev/t/failed-to-count-tokens-for-this-attachment-error/34904/11)

Hope that helps.
