Long context still needs a small working set

Claim

Harness-level compaction and context resets are becoming infrastructure. Both labs now compact server-side with cache-safe designs. Long context is cheap (1M tokens at no premium on Claude 4.6-plus) but quality still degrades with length, so the practice is hybrid: stable shared corpora in long context with caching, large or fast-changing corpora via retrieval.

Evidence

• Anthropic API: context editing clears old tool uses (default trigger 100k tokens, keep 3) and thinking; server-side compaction has a threshold mode and an on-demand mode that keeps recent turns verbatim and runs in the background; client-side compaction control in SDKs is deprecated. Launch post: context editing plus memory tool improved an agentic-search eval by 39% and cut tokens 84% over 100 turns.
• OpenAI Responses: context_management.compact_threshold compacts in request; /responses/compact is a stateless call returning an encrypted, non-readable compaction item to pass forward as-is.
• Claude Code: clears older tool outputs first, then summarises; re-injects CLAUDE.md, memory, skills and plan from disk; re-reads up to five recently touched files; 1M models compact at about 967k; aborts with a thrashing error if one oversized output refills context. Cache is prefix-matched; invalidators include model or effort switch, MCP changes when tools are loaded up front, and compaction; 1-hour TTL on subscription plans, 5 minutes otherwise.
• 1M context GA (2026-03-13) at standard prices; Opus 4.6 scores 78.3% on MRCR v2; yet the compaction docs say quality degrades as a conversation grows.
• Chroma context rot (2025-07-14), 18 models: performance falls with length even on trivial retrieval; worse with low needle-to-question similarity and with distractors; shuffled haystacks beat ordered ones for all 18 models; focused 300-token prompts beat full 113k prompts on LongMemEval.
• Anthropic's harness resets for Sonnet 4.5 became dead weight on Opus 4.5; Managed Agents keeps the full event log so compacted content stays retrievable.

Open questions

• Reported change of the default API cache TTL on 2026-03-06 (single unofficial post). UNVERIFIED.

Sources

• https://platform.claude.com/docs/en/build-with-claude/compaction — docs, Wed Oct 21
• https://platform.claude.com/docs/en/build-with-claude/context-editing — docs, Wed Oct 21
• https://developers.openai.com/api/docs/guides/compaction — docs, Wed Oct 21
• https://code.claude.com/docs/en/context-window — docs, Wed Oct 21
• https://code.claude.com/docs/en/prompt-caching — docs, Wed Oct 21
• https://claude.com/blog/1m-context-ga — blog, Thu Mar 12
• https://www.trychroma.com/research/context-rot — blog, Sun Jul 13

AI-assisted research: gathered and summarised with AI from the sources above, each claim checked against them.