Hello,
I’m making LLM neutral desktop AI agent.
One of its features it’s picking a specific LLM for each task (or subtask inside one large task that was decomposed down).
To do so agent uses several parameters and now relies on historized data or initial manual input. So, initial fresh state of the system is absolutely neutral.
First sorting is simple - after first API call, LLMs discovery and smoke-test the system omits dead or not compatible LLMs. But doesn’t rate them against some common tasks.
Here’s my question:
do you have any formal field in API response to classify the LLM? Like - “this one is for programming”, “this one for deep research”, “this one for image generation”?
It would be great if yes because it would allow to assign a specified LLM for the specific tasks from the beginning.