I’ve noticed a potential discrepancy in input token counting for multiple rounds of chatting. In Google AI Studio’s Playground, the displayed cost estimation seems to only count user’s prompts as input tokens. However, when using API, my understanding is that all previous conversation history, including both user and model tokens, are counted as input in each round of chatting. I’m I missing something?
Hey Steven, agreed on this. I proposed the following to the team:
- We have a single request cost estimate, what would it cost to recreate in a single turn exactly what is shown in the chat UI.
- We have a multi-turn request estimate, if this was an iterative conversation, here’s how much it would have cost
Will get this actioned so that it is more clear!
Hey Logan, thanks for the reply - that’s sounds good and I look forward to that!
A follow-up question: I notice that there are many encrypted thought tokens, which are counted as part of the output tokens, while using API. I wonder how are those tokens handled in the AI studio’s cost estimator?