Alright, this is just frustrating!!
Gemini is only able to use in API mode if you don’t want to leak your data to google!
So I need to buy a chat client, however, when I send 2M token using TypingMind, it always timeout!
And it’s even more ridiculous is GCP is still charging me!!!
I don’t mind losing some money, however, I need a god damn offical chat client for 2m token which don’t leak my company data!!!
Hi @rockmandash ,
You’re right, TypingMind often times out with Gemini 2M because it doesn’t handle large responses well.
Use the Vertex AI SDK directly with streaming to avoid timeouts.
To protect data, don’t use hosted clients self host TypingMind or use your own UI.
GCP charges for tokens sent, even if it fails you can raise a refund request with support.
Currently, there’s no official chat UI for 2M token use best option is using SDK or building your own small interface.
1 Like
Thanks man, hope sombody built a chat ui using Gemini’s official SDK instead of some kind of openai api compaitable approach.
1 Like