Does function calling charge for input tokens on function response?

Gemini API
, ,
1

When I respond with the output of a function, am I charged again for the input tokens in the prompt?

I couldn’t find this clearly spelled out in the docs

2

@dc_l ,

any text/input tokens you send to the model will cost you tokens.

when you call the respective function and send the function response to the model the responce is treated as the input and will cost you tokens.

3

@Akhilesh_Kambhampati ,

Thanks for the reply! I was more wondering about the already existing prompt and system instructions. Are those counted twice, or do I just pay for the new response input?

4

You pay for whatever you send (and the response) when you call generateContent or similar. So if that call contains contents that includes a full content history - the final entry being your function response, then yes, you pay for it all.

An easier way to think about this is that the model retains absolutely nothing about your request. It takes the input, processes it and returns a response. It has no recollection of the request you made prior to this. Everything it can infer it does from the context you give it in that call. The model is stateless.

You can take advantage of context caching to help costs, especially if you’ve a large set of system instructions.

1 Like