The question is whether it is possible to use google coral for relatively large text models like LLAMA. use coral to generate messages, and ultimately communicate with a person in real time.
Hi @Fedorr ,
Google Coral devices are not ideal for running relatively large text models like LLAMA,Gemma,any other Large Language Models for real-time communication due to limitations in processing power and memory. Coral devices are primarily designed for tasks like image classification, object detection, and audio classification but not sure with recent development in the AI that they can support LLM’s models due to limitatioons in Model Size.
May be a good choice if the LLM model is optimized within the Google coral capcity or the cloud services preferable.
Thanks