Using gemini-3.1-pro-preview for evaluation of audio and saw a significant divergence in my score, the same set of audios that used to get around 23% score got 58%, 2.5 pro is still around 25%(expected deviation is atmost 5% points). This significantly affects my evaluation setup.
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Did gemini-3.1-pro-preview change in July 2026? Our results got less consistent | 0 | 68 | August 6, 2026 | |
| Gemini-3.1-pro-preview started failing on April 15, 2026 with 503/504 after ~2 months of stable usage | 2 | 625 | May 12, 2026 | |
| Gemini 3 Pro Preview is not working properly | 1 | 229 | December 23, 2025 | |
| Audio Bug by Priority PayGo Mode | 0 | 39 | March 14, 2026 | |
| Gemini 3 Pro Preview vs. 3.1 Pro Preview | 0 | 114 | March 26, 2026 |