Long-Horizon Task Planning for Computer-Use Agents
Separating task state from execution context prevents errors from compounding across steps.

Long-horizon computer-use agents fail because errors compound across dependent steps, not because any single step exceeds a model's capability. The architectures that actually hold up under multi-step execution share one design principle: task state must be tracked outside the execution context, written to durable storage, and verified independently of the agent's own account of what it did. Deploying a persistent agent per user at scale means giving each one an isolated environment where that state survives restarts, errors, and session breaks, rather than living inside a context window that degrades as it grows. The rest of this piece works through why that principle emerged, how two recent harness designs put it into practice, and what infrastructure it demands underneath.
Why long-horizon computer-use tasks fail at a structural level
A model can execute each individual step of a multi-step task competently and still fail the task as a whole, because the failure accumulates across steps. LongHorizon-Harness, developed at Alibaba, names three failure modes, and they interact to produce this outcome. The first is compounding errors and goal drift: an early mistake distorts the choices that follow it, so the agent's trajectory bends further from the correct path with each additional step. The second is context rot, where information that was relevant early in a task becomes harder for the model to retrieve and use as the interaction history grows longer. The third is task-state loss, in which the agent simply loses track of what it has already finished and what remains open.
None of these three failure modes resolves the others. The three compound together, and that combination is what makes long-horizon execution unreliable even when the underlying language model performs well on isolated tasks. The failure here is structural, built into how state and error propagate through a long task, not a symptom that better prompting or a stronger base model will fix on its own.
Why computer-use tasks are harder to recover from than other agent domains
Computer-use tasks amplify this compounding problem because the agent's only window onto the environment, typically a screenshot, is partial, while the task is ultimately judged by the actual state it leaves behind in files, applications, and operating system resources. A trajectory can end with a clean, convincing final screenshot and still have broken a constraint or left behind a side effect that only a deeper look at the environment would catch.
AgentHorizon, a benchmark of 1,373 computer-use tasks built from human-recorded trajectories, demonstrates how deep this problem runs, even for the systems meant to catch it. That is the same problem the agent itself faces, pushed one level up: judging progress from appearances that don't reliably reflect what actually changed. OneDayAgent, built by researchers at Zhejiang University and Ant Group, names a related failure specific to tasks that cross tool boundaries: an agent moving between web, file, and computation environments can lose formatting constraints or earlier search evidence along the way, so the final artifact quietly omits work the agent already did. OneDayAgent calls this state transfer failure, and it has no real counterpart in single-environment agent domains.
The wrong architecture: tracking state inside the execution context
The root cause behind most of this failure is architectural: existing agent harnesses keep task execution, task state, and completion assessment inside the same growing context, so the agent ends up judging its own progress using the very context its earlier errors have already corrupted. LongHorizon-Harness states this directly. When execution and state management share one growing context, the state becomes hard to track, and a wrong self-assessment at one point in the task propagates forward into every decision that follows it.
Context rot compounds the problem. An agent reporting its own task as complete is not a trustworthy signal under these conditions, because it is drawing that conclusion from a context that may already be degraded and biased by its own prior mistakes. Verification has to come from somewhere outside that context to mean anything.
A natural objection follows: frontier models now ship with very large context windows, so why not just let the agent hold more state in context and call it solved? The window itself is not the bottleneck. LongHorizon-Harness cites evidence that performance drops off because retrieving and correctly using relevant information gets harder once the history accumulates past a certain point, not because the model hits some hard length limit. That is a retrieval problem, and a bigger window does not make retrieval easier; it just gives the model more material to search through.
The shared principle behind harnesses that work: state outside execution, verified independently
Across the harnesses that show consistent gains on long-horizon benchmarks, one design principle keeps reappearing: task state lives explicitly outside the execution context, and it only gets updated with facts the harness has independently verified from the environment, never from the agent's own narration of what it did. LongHorizon-Harness reformulates the entire problem around this idea. State sits outside execution, and the only information that carries across rounds is the task state itself plus audit reports, facts certified by a read-only auditor after each step rather than claims the executing agent makes about itself.
OneDayAgent arrives at a structurally similar answer by a different route. Its execution memory compresses observations and checkpoints subtask state as context pressure builds, and a separate global verification pass re-aligns the finished deliverable with the task's original intent. Both of those operations sit outside the step-by-step executor, not inside it. The practical payoff of this separation is that each role in the harness can run with a context sized to its job: the manager plans from audited facts rather than raw history, the executor acts on a current subtask without needing the full back story, and the verifier checks the result without having been part of producing it. That separation is also where infrastructure starts to matter. Platforms like Maritime run each agent in its own micro-VM with persistent storage, so they enable this separation at the infrastructure layer, and developers can implement the auditor-verified state pattern without building custom isolation and storage systems from scratch.
How the Manage-Execute-Audit loop puts that principle into practice
LongHorizon-Harness's Manage-Execute-Audit loop, or MEA, is the clearest existing implementation of this principle in a working system. Each of its three roles runs in its own bounded context, and state crosses from one round to the next only as verified facts, never as raw execution history. A read-only auditor checks the resulting environment state before the next round can begin, and its reports are the only thing that persists across rounds.
The fresh-context executor matters because it is what keeps context rot from poisoning the agent's decisions: since the executor never sees the accumulated history, it can't be misled by errors baked into that history, and it works only from the audited state summary plus its current subtask. The read-only auditor matters for a different reason. Because it played no part in the execution it's checking, its judgment about what actually changed in the environment isn't colored by what the executor believed was happening. A lightweight AgentAdapter lets these three roles swap across different model and harness backends without touching their native agent loops, making the architecture backend-agnostic.
The measured results bear this out. The same base model, with the same weights, performs dramatically differently depending on whether it operates inside a single growing context or inside the MEA structure.
OneDayAgent: Extending the Principle to Cross-Environment, Multimodal Tasks
OneDayAgent takes the same externalized-state principle and applies it to a messier category of task: open-ended requests that are long-horizon, cross-environment, and multimodal, spanning work across web search, academic search, computation, and file environments. Here the defining failure isn't just context rot within a single session; the agent also loses intermediate state the moment it switches from one tool to another.
OneDayAgent targets three failure modes together rather than one at a time: goal drift, where accumulated constraints pull the agent away from its original intent; context accumulation, where growing history buries earlier constraints the agent still needs; and state transfer failure, where evidence gathered in one environment disappears when the agent moves to the next. A global verification and repair pass then re-aligns the finished deliverable with the original request and patches any localized defects, and it sits outside the step-by-step execution loop, just as LongHorizon-Harness's auditor does.
On AgentIF-OneDay, a benchmark of 104 tasks, OneDayAgent with the GLM-5.2 backend reaches a new state-of-the-art overall score of 0.821. The harness also runs stably across five backend LLMs drawn from three different model families without any backend-specific tuning, a generalization result that echoes LongHorizon-Harness's AgentAdapter finding. Both systems point toward the same conclusion: the architectural pattern is doing the work, not the particular model sitting behind it.
Why verification is harder than it looks
Independent verification, the audit half of the principle, is itself far from solved. Even judges built specifically for this purpose struggle to tell a genuine success apart from a plausible-looking failure, largely because the evidence of what really happened is often buried deep inside a long interaction history, not visible on the surface. AgentHorizon tests a range of judges against its 1,373 computer-use tasks, drawn from human-recorded trajectories across three operating systems. Its paired design, where closely related instructions get swapped to build negative tasks, isolates precisely how good a judge is at catching subtle failures.
The best agentic judge AgentHorizon evaluated, GPT-5.5, still gets roughly one in five judgments wrong even on these carefully constructed tasks. No judge available today is reliable enough to run unsupervised. AgentHorizon also finds that judges vary widely in how well they accept valid trajectories versus reject failed ones, and that tool use helps some models while actually hurting open-weight models, so the architecture of the judge matters as much as which model sits behind it.
The design consequence follows directly: a judge that only reads back over a trajectory passively may not be enough. AgentHorizon's findings point toward judges that can actively locate and verify evidence buried in long interaction histories, checking file state, DOM state, and application backends directly. That connects to a point LongHorizon-Harness surfaces from earlier work on state-tracking agents: a desktop task is judged by the state it actually leaves behind, and a screenshot alone may never reveal that state. The auditor has to read the environment itself, not just interpret the visual trail the agent leaves behind.
What persistent, isolated infrastructure enables that stateless execution cannot
Both the MEA loop and OneDayAgent's checkpointing assume something about the infrastructure underneath them: that state can be written durably, read independently, and restored reliably across execution rounds. Stateless or shared-container infrastructure cannot guarantee any of that. The auditor in the MEA loop has to read files, application backends, and DOM state between execution rounds. The agent's working environment therefore has to persist in a known, inspectable condition between those rounds.
The same requirement appears in OneDayAgent's cross-environment problem. When the agent writes a file, downloads a resource, or produces an intermediate computation in one tool environment, that artifact has to survive the switch to a different tool. That's a persistence guarantee, something stronger than a session guarantee that only promises continuity for as long as a single connection stays open. Execution memory and subtask checkpointing need a durable place to write: not the in-context scratchpad, since that is ephemeral and subject to the same context rot described earlier, and not a remote database call either, since that introduces latency and new failure points into the critical path of execution.
Persistent, isolated compute becomes an operational requirement here. If an agent must survive session breaks, recover cleanly from failures, and maintain an audit trail across a long task sequence, it needs infrastructure that treats per-agent state durability as a given, not something bolted onto a stateless compute layer. Maritime's architecture runs each agent in its own Firecracker micro-VM with a persistent disk, a real kernel, and snapshot-restore that wakes in about a second. The environment state between MEA rounds is simply the VM's own disk and memory, readable by the auditor without re-executing anything. Persistent, per-agent state storage makes that baseline available.
Practical guidance for developers building long-horizon agents today
If you build agents meant to run long tasks reliably, treat state management as a first architectural decision, not a patch you apply after the agent starts failing on multi-step work. Separating the manager, executor, and auditor roles, even in a lightweight form, where a manager writes state to an external store, an executor reads only its current subtask, and a verifier probes the environment rather than reading the executor's own account of events, removes the self-assessment bias that drives cascading failure in single-context harnesses.
Verification strategy should match the environment the agent actually touches. Context pressure deserves attention from the outset. OneDayAgent's execution memory, which compresses observations and checkpoints subtask state before context overflows, functions as a structural requirement for any task spanning hours or multiple environments, not as an optional efficiency gain.
Infrastructure choices should follow from what the harness design actually requires, not the other way around. A harness that checkpoints state between rounds needs somewhere to write that state that survives across rounds, can be read independently of the executor, and restores quickly when an agent resumes. Finally, benchmark the harness independently of the model sitting behind it. Both LongHorizon-Harness and OneDayAgent show consistent gains across multiple backend models, so the harness architecture is the primary lever; test it in place before you tune the model, so you don't mistake a model swap for an architectural fix it never was.
Sources
- LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks
- OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents
- [2610.11050v1] AgentHorizon: Evaluating Agentic Judges for Long-Horizon Computer-Use Tasks
- [2606.29537] OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks


