Context Rotation and Session Handoff in Long-Running AI Agents

Executive Summary

Every autonomous agent built on a transformer eventually hits a hard wall: the context window. Production systems now treat this not as an edge case but as a routine operational event — something to detect, checkpoint, and recover from, the same way distributed systems treat node failure. Three broad strategies have converged across the industry in 2025–2026: server/client-side compaction (summarize the transcript, keep working), externalized memory files (write durable state to disk/DB, re-read on resume), and durable-execution replay (treat each step as a checkpointed unit with explicit side-effect safeguards). No framework relies on a single one of these — Anthropic explicitly recommends combining compaction with the memory tool, and Anthropic's own "harness engineering" guidance for long-running coding agents leans almost entirely on git commits and structured progress files rather than context tricks at all. The hardest unsolved problems are not mechanical (how to serialize state) but epistemic: agents that survive a rotation can carry forward stale or confabulated beliefs about what they already did, and side-effecting tool calls made near a rotation boundary create real double-execution risk unless the actual external-effect boundary records completion or enforces idempotency.

The Problem

A context window is finite (128K–1M tokens depending on model/provider), but an autonomous task — a multi-day coding effort, a standing operations agent, a research loop — is not. Two distinct sub-problems fall under "rotation":

In-session degradation before exhaustion. Chroma's controlled study evaluated 18 LLMs, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3, and found that accuracy generally degrades as input length grows, often in non-uniform and model- or task-specific ways, well before the hard token limit (Chroma, "Context Rot"). The study discusses how input structure may affect attention but does not establish one causal mechanism. Treating this observed degradation as a reason to rotate proactively is therefore an engineering inference, not a causal conclusion from the study.
Hard exhaustion / process handoff. When the transcript can no longer grow, something must give: summarize it, externalize it, or start a new process that reconstructs equivalent state from durable storage.

Both sub-problems force the same design question: what must survive the transition, and what is safe to throw away?

State of Practice: A Framework Survey

Claude Code / Claude platform (Anthropic). Anthropic ships three complementary mechanisms with different scopes. For most long-running conversations and agentic workflows, server-side compaction is the recommended primary strategy: when a configured trigger is crossed, the API summarizes the earlier conversation into a compact block and continues from it (Anthropic, Compaction docs). Context editing is the fine-grained option for removing selected old tool results or thinking blocks when an application needs more control over exactly what leaves the active window (Anthropic, Context Editing docs). Layered on top is the memory tool: a client-executed file interface (/memories/*) for information that must survive beyond the active transcript (Anthropic, Memory tool docs). Anthropic recommends combining compaction with memory rather than treating either as a complete substitute for the other; actual trigger points and compression ratios remain workload- and configuration-dependent.

Anthropic's long-running-agent harness pattern. For genuinely multi-day autonomous coding work, Anthropic's engineering team bypasses in-context tricks almost entirely in favor of filesystem/git state. An initializer session sets up init.sh, a claude-progress.txt log, and a JSON feature checklist (200+ items in their claude.ai-clone case study) where each subsequent session may only flip a passes: true/false boolean — explicitly forbidden from editing or removing test definitions, "because this could lead to missing or buggy functionality." Every new session runs a fixed startup ritual: pwd, read git log, read the progress file, pick the highest-priority unfinished feature, boot the dev server, run one verification test — before doing anything else (Anthropic, "Effective harnesses for long-running agents"). Git commits function as the actual checkpoint boundary; the progress file is a human/agent-readable index into that history.

LangGraph (LangChain). Checkpointing is a first-class graph primitive: every superstep persists a full state snapshot keyed by thread_id, enabling resume after interruption, human-in-the-loop pauses, or crash recovery (LangChain, Persistence docs). The default InMemorySaver/MemorySaver is explicitly non-durable — state vanishes on process restart — and production deployments are directed to PostgresSaver/SqliteSaver. A documented tradeoff: because a checkpoint is written per superstep, long-running agents accumulate large checkpoint volumes, requiring explicit pruning/retention policies. LangGraph separates short-term memory (checkpointer, thread-scoped) from long-term memory (a cross-thread store), mirroring the working-memory/episodic-memory split seen elsewhere.

Letta (formerly MemGPT). Pioneered the "LLM as OS" framing: a three-tier hierarchy of core memory (small, always in-context, agent-editable "RAM"), recall memory (searchable full history, "disk cache"), and archival memory (long-term store queried via tool calls) (Letta, "Memory Blocks"). Memory blocks are individually persisted with a stable block_id; the active context window is "compiled" from current DB block values on each request rather than replayed from a message log. Notably, Letta favors explicit, tool-mediated editing over automatic overflow summarization — the agent itself decides what to page in or out, rather than a framework-level trigger doing it silently.

OpenAI Agents SDK. Handoffs default to model-visible transcript inheritance — "it's as though the new agent takes over the conversation, and gets to see the entire previous conversation history" — with input_filter and a beta nest_handoff_history mode available to reshape that input (OpenAI Agents SDK, Handoffs docs). This is distinct from RunContextWrapper.context: the latter is caller-supplied local state and dependencies shared with agents, tools, and handoffs during one Runner.run() invocation, and is not sent to the model (OpenAI Agents SDK, Context management). Neither mechanism by itself is cross-run application persistence; state that must outlive a run needs an application-owned durable store, while transcript continuation can use an SDK Session or an equivalent history store.

AutoGen (Microsoft). State handling is comparatively primitive: save_state/load_state serialize message threads and group-chat manager state to arbitrary JSON, with no native crash-recovery or automatic checkpointing layer — open GitHub issues have requested runtime-level checkpoint/rollback for years without being fully resolved (AutoGen docs, Managing State; autogen#3619).

OpenHands. Built around a single mutable ConversationState object as the source of truth, with a complete event log (user messages, agent actions, system events) persisted under a persistence_dir + conversation_id, restorable from a different process (OpenHands SDK, Persistence docs). ACP-style conversations also persist the wrapped tool's session ID across agent-server restarts. When a recycled cloud sandbox has lost base_state.json, the caller can mirror that ID in an external durable store, feed it back through acp_resume_session_id, and let OpenHands call ACP session/load; an unknown or unloadable ID falls back to a fresh session (OpenHands ACP durable-resume implementation, pinned revision). Resume therefore crosses the sandbox boundary only if both the session ID is durably retained and the ACP server still retains the corresponding session.

Durable-execution substrates (Temporal, Inngest, Restate). Increasingly used as the reliability layer underneath agent frameworks, but their contracts are not interchangeable. Temporal reconstructs workflow state by replaying a durable Event History after failure (Temporal, Event History). Inngest memoizes state by named step ID, retries a failed step independently, and avoids re-running successful steps; it recommends placing non-deterministic side effects inside step.run() so their results are checkpointed (Inngest, Steps). Restate journals operations and their results, skips completed journal entries during retry, and separately offers request deduplication when the caller supplies an idempotency key (Restate, journal replay; request deduplication). These mechanisms protect only effects routed through their recorded boundaries; an unjournaled or uncheckpointed external write still needs the target system's idempotency or an application-owned effect ledger.

Handoff-Content Taxonomy: What to Carry, What to Drop

Must survive:

Active task definition and success criteria (the equivalent of Anthropic's feature checklist)
Pending obligations — outstanding tool calls awaiting results, unacknowledged user requests, scheduled follow-ups
Irreversible-decision log — architectural choices, constraints already ruled out, and why (so they aren't silently re-litigated)
Side-effect ledger — what external actions were already taken, with idempotency keys, to prevent re-execution
Channel/conversation identity — which user, which thread, which reply path, so continuity is externally invisible

Safe to drop or compress:

Raw tool outputs already synthesized into a conclusion (large file reads, search results, stack traces once diagnosed)
Redundant back-and-forth that converged on a decision already recorded elsewhere
Full reasoning/thinking traces once their conclusion is captured (Claude's clear_thinking edit targets exactly this)
Exploratory dead ends, provided the fact that they were dead ends (not just their content) is preserved — dropping the content but losing the "already tried, failed" fact is a major source of repeated wasted work

The dividing line, per Anthropic's context-engineering guidance, is roughly: keep anything whose absence would be undetectable until it caused a wrong decision later; compress anything whose value was already fully extracted.

Failure Modes

Lost pending items. Summarization is lossy by construction; Anthropic itself warns "overly aggressive compaction can result in loss of subtle but critical context whose importance only becomes apparent later" (Anthropic context-engineering blog). A rotation that summarizes away an open question or unconfirmed commitment silently drops it.
Double execution / non-idempotent replay. When a handoff or crash recovery retries work whose external effect (an email sent, a record written, a payment) already fired but was not durably recorded, naive retry can duplicate it. Inngest avoids re-running completed step.run() calls whose results were checkpointed, while Restate skips completed journal entries and can deduplicate duplicate requests carrying an idempotency key; neither promise automatically covers a write made outside those boundaries (Inngest, step checkpointing; Restate, request deduplication).
Stale self-belief and memory confabulation. Long-lived memory stores accumulate assertions that were true once but no longer are, and — worse — agents can write confidently incorrect accounts of what they did into memory, then act on that false record across resets. The STALE benchmark specifically probes whether agents can detect that a stored memory is invalidated by new evidence (STALE, arXiv 2605.06527); separate work on memory confabulation shows these false beliefs can be self-reinforcing, since an agent that "concludes" a path always fails may never revisit it (arXiv 2605.29463).
Premature victory / abandoned partial work. Anthropic's harness postmortems name this explicitly: agents mistake partial progress for completion, or abandon a half-finished feature mid-session with no artifact marking it incomplete — solved only by structured, machine-checkable state (the passes: true/false checklist) rather than prose summaries.
Cross-process handoff loss at tool boundaries. OpenHands' ACP resume path shows that handoff fidelity is only as strong as the weakest persisted layer: after sandbox recycling, the application must durably mirror the ACP session ID and the ACP server must still be able to load it. If either side is missing, OpenHands deliberately falls back to a fresh session rather than claiming continuity.

Design Patterns and Recommendations

Rotate before exhaustion, not only at it. Because context degradation is real but non-uniform across models and tasks, set a proactive policy from measured answer quality, cost, and remaining headroom for the actual workload rather than copying one universal utilization percentage.
Separate mechanism by content type. Use compaction as the primary strategy for most long-running conversational state; use context editing when an application needs fine-grained control over bulky, already-synthesized tool or thinking content; use externalized files/DB for anything that must survive process death, not just window rotation.
Make state machine-checkable, not prose-summarized, wherever correctness matters. A boolean checklist or structured JSON progress file is far harder to hallucinate past than a paragraph summary.
Give every side effect a stable execution identity and a durable boundary. Route it through a checkpointed step or journaled operation when possible, and use the target system's idempotency key or an application-owned effect ledger when it is not. Workflow replay alone is not an end-to-end exactly-once guarantee.
Record what was tried and failed, not just what succeeded — the negative-result log is what prevents repeated wasted exploration after a reset.
Treat sub-agents as a context-isolation primitive, not just a parallelism one. Returning a 1–2K token distilled summary from a 50K-token sub-exploration is itself a rotation strategy, applied continuously rather than at a threshold.
Verify on resume, don't just believe the checkpoint. Anthropic's startup ritual (re-run a smoke test before continuing) guards against a checkpoint that was itself written from a stale or wrong belief.

Open Questions

The field has converged on mechanisms faster than on epistemics. It remains unresolved how an agent should detect, at resume time, that its own carried-forward beliefs are stale rather than trusting its last checkpoint by default (STALE and confabulation research are early, not yet production-standard). There is no widely adopted cross-framework standard for a handoff envelope — every framework (LangGraph threads, OpenHands conversation state, Claude Code progress files, Letta memory blocks) uses an incompatible bespoke format, which blocks portability of a running agent between runtimes. And the idempotency story for probabilistic side effects — where a retried LLM call may legitimately produce a different, not-obviously-duplicate action — is still handled ad hoc rather than with a general primitive. Durable workflow replay is mature, but it does not by itself settle idempotency at every external-effect boundary.