I’m constantly finding myself forgetting to switch the model and sending really simple tasks to Argon / Carbon or really complex tasks to Flash. It would be nice if “Auto” were an option and some lightweight model (like Flash-Lite) could assess the prompt and automatically switch the model given the context.
Even better if you could configure auto to select from a specific set of models and efforts. I was suggesting to use Flash-Lite as a classifier because it’s cheap and very fast, but I could also imagine that Lite is perfectly fine at answering some very basic questions and it would come back with an answer much faster and with less cost.
I guess one thing to be aware of is that switching models constantly would require cache rebuilding which could work against you in cost. But I’d rather have code figure all that out for me than doing it myself.