Does the OpenAI format support context caching?

Gemini API
,
1

Hello,

The new OpenAI API format is very convenient – thank you for adding it! Does it support context caching?

I was unable to find the answer from (a) the docs, (b) searching this forum, and (c) the internet, so I am asking here.

Thanks!

3

Yes, it’s documented here:

Ahh, sorry, that’s for the Gemini API directly, you are referring to the OpenAI API.

No, then I haven’t seen that feature yet. Perhaps it’s on the roadmap of the compatibility. Right now, I guess we have to be patient.

1 Like
5

Hi @lmk

The OpenAI API does not provide explicit context caching. However, caching is automatically applied for prompts that are 1024 tokens or longer. If you need explicit context caching, you can consider using the Gemini API.

Thanks