If it’s a lower level or more routine type of task or request I have been doing similar to what you are building using 3.8 flash first and then going over things with 4.0.
With higher level systems planning, and critical tasks, I’m using 4.0 first primarily, and then for secondary tasks related to these high level projects I’m using lower tier models to save tokens as appropriate.
It’s kind of a modulation thing, like I’ll ask myself do I need the more powerful more capacity for this specific prompt or this stage of workflow, if not I’ll switch to lower models as appropriate for the tasks.
I think it’s possible to use too big of a model for some things and it could actually cause problems of some sort of overfitting if you will.
In my opinion sticking with one model seems to be like sticking with one opinion, it can only get you so far. And how you integrate various opinions is for your own interpretation.
I think as time progresses there will definitely be some model specific task and workflow delegation hierarchies that are widespread.
Yeah, me too. It has to work like that because as models get larger, their costs go up, so companies have to take this approach.
Regarding your idea about overfitting, I disagree. But we could say that Pro models take more time and think a lot about edge cases before deciding, whereas Flash models seem to think less. It all comes down to this: if you have a simple task and need it done quickly, use Flash models; otherwise, Pro models are there for you. Overfitting is a concept specific to training ML models. Maybe you’ve been studying ML lately!
Yeah, I definitely used the wrong term, maybe oversized or overkill is more appropriate. What I was trying to get at, is that you can potentially cause some issues with too big of a model for a simple task. Ending up with excessive details and suggestions that need to be sorted through. Of course spending too much tokens for the objective is the real issue.
What gets really impressive fast is stacking distilled and quantized models across your operating infrastructure so you can route coding resources to the proper models which handle each need more effectively with more precision. Highly recommend implementing Google GenKit if you haven’t yet and setting up your own sovereign Git (Forejo or Gitea) if you haven’t already to assist with routing, memory and logging.