Hello,
I need help to choose a tech stack for an application.
It is an application with AI Assistant (an Agent) that reads scripts to users. The Agent should select appropriate script based on user’s input. I am thinking to go either with Mistral-7B-Instruct-v0.2 or llama2 LLM but opened for other suggestions.
The Model should have access to the database to retrieve user’s information and scripts.
What is the best place to host an LLM and build such an application? Which tools shall I use?
Can someone please push me to the right direction.
As per my understanding These tech stacks for your AI-driven script-reading application is crucial for its performance, scalability, and maintainability.
AI Model Selection: Mistral-7B-Instruct-v0.2 or llama2 LLM ,
Hosting and Infrastructure: Cloud Platforms , Containerization, Serverless Architecture
Database: Relational or NoSQL , Data Security
Frontend Framework: React or Vue.js, Responsive Design
Integration and APIs: RESTful APIs, Authentication and Authorization
Monitoring and Maintenance: Logging and Monitoring, Continuous Integration/Continuous Deployment (CI/CD)
Since your application needs an AI Assistant to read scripts to users and select the appropriate script based on user input, using models like Mistral-7B-Instruct-v0.2 or LLaMA2 LLM are good starting points.
For hosting the LLM and building your application, you might consider the following:
Cloud Providers: AWS, Google Cloud Platform, and Azure are popular choices for hosting machine learning models. They offer robust infrastructure, scalability, and integrated AI services.
Database Access: Ensure your chosen model can seamlessly interact with your database to retrieve user information and scripts. Consider using managed databases like Amazon RDS or Google Cloud SQL for reliability.
Frameworks and Tools:
Hugging Face Transformers: This library is excellent for working with LLMs and can integrate with various cloud providers.
Flask or FastAPI: For creating the backend API that will handle user input and responses.
Docker: For containerizing your application to ensure consistency across different environments.
For a more detailed guide on choosing the right technology stack for SaaS applications, you can check out this tech stack for saas guide.