Degradation in model quality as the context window fills with irrelevant tokens—the model’s attention spreads too thin over signal and noise.
Bigger context windows do not linearly help. Attention dilution and the related lost-in-the-middle effect mean models often miss information buried between relevant content. Mitigations: tighter context, reranking, summarization.