Is there a way to provide a Tokenize/Detokenize api endpoints?
I am upgrading the gemini binding in lollms and one functionality it can not provide yet is the tokenization. It is useful for lollms’s playground page where users can experiment with AI.
Thanks.