Another rather significant issue has popped up on heavier and longer context testing
There is a significant structural repetition pattern.
In grittier scenes or in longer context in general (50k+ tokens), 3.1 will start stacking 2 adjective sentences. Every paragraph sometimes multiple times in a paragraph.
- Large, bare feet.
- Slick, wet grass.
- Soft, steady cadence.
- Aggressive, possessive state
etc.
2.5 did this but it never started spiraling to near every paragraph. This negatively effects the writing quality pretty strongly.
Within this structural repetition there is also word repetition, the reliance on the word “Heavy” for example, has been just throw around everywhere.
Can’t be controlled well through prompting. Seems to want to heavily default to it after a bit.