Although for most projects 1M tokens context window is enough, some big files analyses can have a huge benefit of a larger context window.
For example, we’ve made a pilot summarizing psychological/medical old records. The normal workflow is to feed the model with one record (of the same patient) at a time, and then do the manipulations and processing in the database/agent/embeddings levels. But for feeding a full patient file into the model for summarization, analysis or background info - 1M is not enough (I find the average file to be 1.5M-1.8M tokens).
The Gemini-2.0-pro (IMNO) had 2M context window, but it’s not available anymore.
To my opinion, a model, even if not the main one, with such a capability, could open large variety of new use cases.
Will really appreciate development teams to look into this issue.