Count_tokens 403 (File API Permission) BUG - tested on Cookbook Colab

Welcome to the forum, Alan. I think the missing field for token count is a bug that has been reported on multiple occasions, like here Why is token_count in GenerateContentResponse always 0?

There is a potential workaround, by using the usageMetadata field, as described in this post:

The field is documented here: GenerateContentResponse  |  Google AI for Developers  |  Google for Developers

I have been unable to solicit official guidance from anyone in Google whether the workaround is expected to become the permanent fix or whether the per-Candidate token count will be eventually introduced into the implementation, aligning the implementation with the documented API.

Hope that helps