count_tokens returns 403 Forbidden Error with File API media inputs. Tried the js library as well as the python lib. Even the Colab cookbook reproduces the error with just an audio file. See image attached. Colab from cookbook to reproduce bug: cookbook/quickstarts/Audio.ipynb at main · google-gemini/cookbook · GitHub
Error: “You do not have permission to access the File [file name] or it may not exist.”
Other uses of File API work fine, I can upload, list, get and delete. Also, I can generate content using 1.5 with one or more of the files uploaded, referencing their URI. Problem is specific to count_token.
This bug shows up for both gemini-1.5-flash and gemini-1.5-pro, using paid or free accounts. Since usageMetadata is currently not returning prompt token count (only candidates count is returned now) this means that there’s no way to find the total tokens used in a prompt that uses FileDataParts.
Please adivse.
Hi Alan
From the image that you used, I can see that it is necessary to replace the expression [your_file] with the name of your audio file. According to the example shown in the cookbook, it would look like this: ‘’‘model.count_tokens(‘sample.mp3’)’‘’.See in this image:
Hope that helps 
Hi Carlos. That’s not what is going on here. sample.mp3 is the name of the sample file in the Colab, that is downloaded, uploaded to Gemini’s File API and stored in a variable called your_file. Let me explain in more detail.
The linked Colab (that is part of Gemini’s cookbook), first downloads JFK’s 1961 State Of The Union speech from a hardcoded URL to a local file and calls it sample.mp3. It then uses the File API’s upload method to upload it to Gemini’s file storage and assigns the result of the upload (an instance of File, see REST Resource: files | Google AI for Developers | Google for Developers) to a local variable called your_file (your_file = genai.upload_file(path='sample.mp3')).
It then asks gemini-1.5-pro to generate a summary of your_file (effectively, it sends an array containing a text prompt and the reference to the uploaded File). After that the Colab asks Gemini to count the tokens of the same your_file file.
The summary is correctly generated, attesting that the issue has nothing to do with the file name, File API upload, variable name or their contents. The problem is with count_tokens.
I hope this helps clarify what I’m reporting.
1 Like
Welcome to the forum, Alan. I think the missing field for token count is a bug that has been reported on multiple occasions, like here Why is token_count in GenerateContentResponse always 0?
There is a potential workaround, by using the usageMetadata field, as described in this post:
The field is documented here: GenerateContentResponse | Google AI for Developers | Google for Developers
I have been unable to solicit official guidance from anyone in Google whether the workaround is expected to become the permanent fix or whether the per-Candidate token count will be eventually introduced into the implementation, aligning the implementation with the documented API.
Hope that helps
Hi OrangeNebula. Thanks for your reply! I’m not really talking about the token_count field in GenerateContentResponse, but about the count_tokens() method (in the REST api: Method: models.countTokens | Google AI for Developers | Google for Developers , in the Python SDK: google.generativeai.GenerativeModel | Google AI for Developers | Google for Developers , and in the JS SDK: generative-ai-js/docs/reference/main/generative-ai.generativemodel.counttokens.md at main · google-gemini/generative-ai-js · GitHub)
The problem with usageMetadata (which by the way was also broken for me until literally 3 minutes ago! - It was not including the prompt tokens, now it is and it’s a great improvement!! Thanks for fixing that!), is that one needs to actually ask Gemini to process, and thus incur both the time and dollar costs associated with it, to get the number of tokens. For both time and financial reasons it may be required to have that info beforehand, like count_tokens is supposed to do. If you have a workaround for that, or if maybe there’s a way to get to usageMetadata without running the prompt that I’m not aware of, it’d be awesome!
1 Like
OK, happy update: count_tokens now started working now too! Thanks for the fix!
3 Likes
True, there was a correction.
Previously this count_tokens() method was only processing text-based data. Even in the example I put with an image, only later did I realize that it actually only counted the number of tokens in the expression ‘sample.mp3’ and not the number of tokens in the audio file. 4 Tokens only! 
But it now seems to have improved, after the quick update the method is also working for media files. I tested it with the audio example and it returned 83552 tokens! More coherent!
I also tried with an image and it worked.
However, I had a question: do cash costs only occur when executing methods such as generate_content() and send_message()? For example, would the upload_file() or start_chat() methods or count_tokens() itself already generate eventual costs in dollars? 
The generate_content() and send_message() methods send tokens to the model and get tokens back - so they incur the per-token cost.
The file operations (such as upload_file()) do not incur any costs. Storage is free.
The start_chat() method only runs locally (it sets up local data structures to store your history in memory) so doesn’t incur any costs.
The count_tokens() method is a bit of an oddity. Although it sends tokens, those do not go to the model, just the tokenizer. It does not incur any charges. (The Billing FAQ makes this explicitly clear as well.)
1 Like