Chapter 8: The Agent Execution Harness — The Runtime That Makes Agents Run

Introduction: The Model Is Not the Agent

Here is a distinction that takes most people a while to internalize: the model is not the agent.

A model takes text in and produces text out. That’s the whole contract. It has no memory of the last turn, no ability to call a tool, no way to know whether its last action succeeded, no concept of “trying again.” Left to itself, a model is a brilliant one-shot function.

Everything that turns that function into something you’d call an agent — the loop that keeps it going, the code that runs the tools it asks for, the logic that decides when to stop, the machinery that remembers what happened so a crashed run can resume — lives outside the model. LangChain has a tidy equation for it:

Agent = Model + Harness

The harness is every piece of code, configuration, and execution logic that isn’t the model itself. It’s the runtime. And it is where the difference between a flashy demo and a system you’d trust in production actually lives. A demo agent needs a clever prompt. A production agent needs a harness that can run for a hundred turns without exhausting its context, recover when a tool times out, persist its state so a deploy doesn’t wipe a long-running task, and emit enough telemetry that you can debug it at 2 a.m.

This chapter is a tour of that runtime. We’ll start with the loop at the center of every agent, then work outward through the seven things a harness has to get right: dispatching tools, managing the context window, detecting when the agent is stuck, persisting state for resumable runs, running work concurrently, and making the whole thing observable. Along the way we’ll look at how the major systems — Anthropic’s Claude Agent SDK, OpenAI’s Agents SDK, Google’s ADK, Microsoft’s Agent Framework, LangGraph, Temporal, and AWS Bedrock AgentCore — actually build each piece, because the interesting engineering is in the differences.

By the end you should be able to look at any agent framework and quickly answer: how does its loop run, where does its state live, how does it keep the context from overflowing, and how would I debug it?

1. The Agent Loop

1.1 The Shared Core

Almost every agent runtime in production runs a version of the same loop, and it traces back to a 2022 paper called ReAct. The idea is to interleave reasoning and acting:

1. Thought — the model reasons about what to do next.
2. Action — it calls a tool.
3. Observation — the tool returns a result, which the model reads.

Then it loops. The thought plans and handles surprises; the action reaches out to the world; the observation feeds ground truth back in. The loop ends when the model produces an answer without asking for another tool.

What’s striking is how little this has changed. Anthropic’s own definition of an agent runtime, written two years after ReAct, is almost a restatement: agents are “LLMs using tools based on environmental feedback in a loop,” gaining “ground truth from the environment at each step … to assess its progress.” When a 2022 academic paper and a frontier lab’s 2024 engineering post describe the same while loop, that’s as close to consensus as this field gets.

The single most important property of this loop is the observation. This is what separates an agent from a model that just talks. When the agent calls a tool and reads the result, it gets a fact it didn’t have before — a file’s actual contents, a test that actually passed or failed, an API that actually returned a 404. That grounding is the whole point. An agent that confidently reports a config value it imagined is a liability; one that reads the file first is useful.

1.2 Reactive vs. Deliberative: The One Big Divergence

The loop is shared. How much the agent plans before acting is where designs split.

A plain ReAct loop is reactive: it thinks one step at a time. This is wonderfully adaptive — the agent responds to whatever the last observation revealed — but it has a well-documented failure mode. A reactive agent that hits a failing tool call “can easily get stuck in a reasoning loop, repeatedly trying the same failed action.” It can’t see the forest because it’s always staring at the current tree.

The alternative is Plan-then-Execute. Here a reasoning-heavy Planner lays out the whole plan up front — invoked sparingly, “perhaps only once at the beginning” — and a cheaper Executor (a smaller model, or even plain non-LLM functions) carries out each step. When something breaks, a re-planning loop reconsiders the whole plan rather than flailing at the current step. The payoff is twofold: far fewer calls to the expensive planning model, and strategic error recovery instead of the stuck-in-a-loop trap.

The trade-off is the usual one. Reactive loops handle surprises gracefully but can lose the thread on long tasks. Deliberative loops stay on-task but are rigid when reality diverges from the plan. Most production systems blend the two — a reactive loop with periodic re-planning, or a planner that delegates reactive sub-loops.

What to internalize: when you pick or build an agent runtime, the first question isn’t “which model?” — it’s “how reactive should the loop be?” Open-ended exploration (research, debugging) rewards reactivity. Well-understood multi-step procedures (data pipelines, form-filling) reward planning.

1.3 How the Loop Looks in Practice

The same loop wears different clothes across frameworks. A few worth knowing:

• The OpenAI Agents SDK centers on a Runner. You call Runner.run() (async), Runner.run_sync(), or Runner.run_streamed(), and it runs the loop: call the model, and either end (the model produced a final answer with no tool calls), hand off to another agent, or execute the tool calls, append the results, and go again.

• Google’s ADK runs a distinctive yield-based loop. A Runner coordinates a single invocation; the agent runs until it has something to report, then yields an Event. The Runner processes that event — crucially, committing any state changes — and only then does the agent resume. The agent literally pauses after each yield until the runtime catches up. We’ll see why that pause matters when we get to state.

• Microsoft’s Agent Framework — the explicit successor to both Semantic Kernel and AutoGen — represents an agent as an AIAgent you create and call with RunAsync(...), and layers graph-based workflows on top for multi-step orchestration. It’s also the one major framework that makes the harness a first-class, named primitive: a HarnessAgent it defines as “the runtime scaffolding that turns a language model into an agent” — driving the model and tool calls, managing context, applying approval policies. The equation this chapter opened with is, for Microsoft, an actual class name.

• AWS Bedrock AgentCore offers the maximally-managed version: a Harness that is “a managed agent loop … invoke[d] with a single API call.” You hand it a model, a system prompt, and tools; it runs the loop as a hosted service. No loop code in your process at all.

That last one frames a spectrum we’ll return to: from you write the loop (low-level SDKs) to the cloud runs the loop (managed services).

2. Tool Dispatch and Result Handling

If the loop is the heart, tool dispatch is the hands. When the model says “call search with these arguments,” something has to parse that request, actually run the search, capture the result, and feed it back in the right format. That something is the harness.

2.1 Who Runs the Tool Loop?

The first design decision is who owns the tool loop — you or the framework. Anthropic draws this contrast cleanly across two of its own SDKs:

• With the lower-level Client SDK, you write the loop: a while response.stop_reason == "tool_use": loop that calls your tool executor and feeds the tool_result back. Total control, more boilerplate.
• With the higher-level Agent SDK, Claude handles it — the SDK runs the loop with built-in tool execution.

This is the same managed-vs-manual spectrum from the loop discussion, now applied to tools. Lower-level SDKs make you wire the dispatch; higher-level ones do it for you. Neither is “better” — it’s a question of how much control you need versus how much boilerplate you want to own.

2.2 Treating Bad Tool Calls as First-Class Errors

A model will, sometimes, emit malformed JSON or call a tool that doesn’t exist. A robust harness treats this as a distinct, catchable condition rather than letting it silently corrupt the loop. OpenAI’s SDK, for instance, raises a typed ModelBehaviorError specifically when “the model produces malformed JSON or invalid tool calls,” sitting under a base AgentsException. The lesson generalizes: give bad tool calls their own error class, so the harness can decide whether to retry, repair, or surface them — instead of crashing or, worse, proceeding on garbage.

2.3 Tools as Retryable, Durable Units

Some systems go further and make each tool call a durable, retryable unit of work. Temporal models individual operations — LLM calls, web searches, API requests — as Activities, each invoked with a per-attempt timeout (e.g. two minutes) and “automatically retried … with configurable backoff, so transient failures don’t derail the entire … session.” This is an important shift in thinking: the model shouldn’t be responsible for retrying a flaky API. The harness should, transparently, below the level the agent even sees.

2.4 MCP: The Tool Layer Is Standardizing

One genuine convergence: the Model Context Protocol (MCP, covered in Chapter 5) is becoming the common way to expose tools to agents. AWS’s AgentCore Gateway converts existing APIs, Lambda functions, and services into MCP-compatible tools; Microsoft’s Agent Framework ships MCP clients for tool integration. If you’re building tools today, building them as MCP servers is the bet most of the industry is making.

2.5 Don’t Let Tool Output Eat Your Context

Large tool outputs are a quiet context killer — a single verbose API response can swallow thousands of tokens. A useful harness trick is tool-output offloading: keep only the head and tail of a big output in context (above some token threshold) and write the full result to the filesystem, letting the model fetch it on demand. The result is handled without being retained in full. This bridges directly into the next part, because managing tool output is really a special case of the central challenge: managing the context window.

3. Context Window Management

3.1 Why Bigger Windows Didn’t Solve This

You might think million-token context windows made this problem go away. They didn’t — for the same reasons Chapter 3 covered with memory. Filling a huge window is expensive, and models don’t reliably use everything in a large context anyway. For a long-running agent, the context fills up no matter how big it is: a hundred turns of tool calls, observations, and reasoning will exhaust any window eventually. So the harness has to actively manage what stays in the window. This is the most active area of harness engineering right now, and three techniques show up everywhere.

3.2 Technique 1: Compaction

Compaction means taking a conversation that’s approaching the window limit, summarizing it, and starting a fresh window seeded with that summary. The art is in what you keep. Anthropic describes preserving “architectural decisions, unresolved bugs, and implementation details while discarding redundant tool outputs.” The five-hundred-line file the agent already finished reading can go; the fact that it found a bug on line 412 must stay.

3.3 Technique 2: Context Editing

Context editing is more surgical: automatically clearing stale tool calls and results from the window as you approach the token limit, while keeping the conversation flowing. By default it clears old tool results (keeping the most recent few) rather than the reasoning around them — the bulky observation from twenty turns ago goes, the thread of the conversation stays.

How much does this buy you? Anthropic reports that in a 100-turn web-search evaluation, context editing let agents finish workflows that would otherwise fail from context exhaustion — while cutting token use by 84% (with a 29% quality lift from context editing alone, or 39% combined with a file-based memory tool).

A caveat worth carrying: that 84% is a vendor’s own internal benchmark — Anthropic calls it “an internal evaluation set for agentic search” — with no published methodology and, as of this writing, no independent replication anywhere outside Anthropic’s own channels. Treat numbers like it as directional evidence that the technique works, not as a guarantee you’ll see the same figure. The mechanism is sound; the exact payoff depends on your workload.

3.4 Technique 3: Sub-Agent Context Isolation

The third technique is the most powerful, and it doubles as a concurrency pattern. Instead of doing everything in one ever-growing context, the agent spins up a sub-agent with its own clean window to handle a focused subtask. The sub-agent can burn tens of thousands of tokens exploring — and then returns only “a condensed, distilled summary of its work (often 1,000–2,000 tokens).”

The main agent never sees the sub-agent’s messy exploration, only its conclusion. This keeps the coordinating context lean while still doing deep work underneath. It’s the agentic equivalent of a function call: the caller doesn’t carry the callee’s local variables around after it returns.

3.5 The Same Lesson from Research

Academic work lands in the same place from the other direction. Context-Folding lets an agent branch into a sub-trajectory for a subtask and then “fold” it when done — collapsing the intermediate steps but keeping a concise summary — and reports matching or beating a ReAct baseline “while using an active context 10× smaller.” A related method, SLIM, periodically summarizes the trajectory to keep things lean, reporting far fewer tool calls on long search tasks. A 2025 follow-on, FoldAct, points at what makes this genuinely hard: every time the agent summarizes and folds, it changes what it will observe next — the summary reshapes the agent’s own future inputs, so a naive fold can quietly degrade later decisions. (All three are recent preprints, benchmarked on specific tasks, and the fold methods lean on a custom training step — treat the magnitudes as suggestive.) Different mechanisms, one principle: keep the active context small. A 200K window full of stale junk performs worse than a 20K window full of relevant signal.

What to fix first: if your agent dies on long tasks, the cause is almost always context, and the highest-leverage fix is usually sub-agent isolation — push exploratory work into a child context and bring back only the summary.

4. Stall Detection and Retry Logic

We’ve met the failure mode already: a reactive agent stuck repeating the same failed action. So how do harnesses notice an agent is going nowhere and intervene? Honestly, less elegantly than you’d hope — this is the least mature corner of the field. The dominant tools are blunt but effective.

4.1 Hard Iteration Caps

The simplest and most universal backstop is a turn limit. OpenAI’s SDK takes a max_turns parameter; exceed it and it raises MaxTurnsExceeded. That’s it — bound the loop, fail loudly, don’t let a confused agent spin forever burning tokens. Google’s ADK has the same idea for its deterministic LoopAgent, which stops after max_iterations or when a sub-agent explicitly signals it’s done (escalate = True); notably, “the LoopAgent itself does not inherently decide when to stop looping.” Knowing when to quit is the harness’s job, not something you can assume the model will figure out.

4.2 Push Retries Below the Agent

For transient failures — a rate limit, a flaky network call — the right move is to retry below the agent’s awareness. Temporal’s model is the clear example: each Activity carries a per-attempt timeout and is retried with configurable backoff automatically. The agent never sees the blip. This is the production-grade version of resilience: don’t ask the model to handle exponential backoff — that’s the runtime’s job. The model should only see a tool failure when it’s a real failure worth reasoning about, not a transient one the harness could have absorbed.

4.3 Self-Verification Hooks

A more proactive approach is to prevent stalls by forcing a verification step. Harness hooks can run a predefined check — say, a test suite — and “loop back to the model on failure with error messages.” Instead of detecting a stall after the fact, you build a checkpoint into the loop: write code, run the tests, and if they fail, feed the errors back so the model fixes them. The agent can’t declare victory on a broken build.

The honest summary: stall handling today is mostly hard limits plus typed errors plus self-verification checkpoints. Sophisticated loop-detection — noticing the agent is semantically going in circles — is still largely an open problem. If you’re building a harness, start with a turn cap. It’s crude, but it’s the difference between a bounded failure and a runaway bill.

5. Session State and Journaling

An agent that forgets everything when the process restarts isn’t much use for long-running work. Session state is how a harness persists what’s happened so a run can be resumed, forked, or audited. This is where the major systems diverge the most — and the differences are real architectural choices, not cosmetic API differences. Five distinct models are worth knowing.

5.1 Anthropic: JSONL Files You Can Resume or Fork

The Claude Agent SDK — renamed from the “Claude Code SDK” in late 2025, once Anthropic decided the same harness was for building any agent, not just coding ones — persists session state as JSONL on your filesystem. You capture a session_id, and later you can resume that session, or fork it “to explore different approaches” — branching a conversation to try two directions from the same starting point. Simple, local, inspectable: you can literally open the file and read what happened. (Anthropic-hosted Managed Agents instead keep an Anthropic-hosted event log.)

5.2 OpenAI: Pluggable Session Backends

OpenAI puts a Session abstraction in front of storage. You pass a session= to Runner.run() and history management is automatic — the runner retrieves prior history before each run and stores new items after. What’s nice is that the backend is swappable behind one protocol:

Same interface (get_items, add_items, pop_item, clear_session), different durability — from throwaway in-memory to production database to fully server-managed.

5.3 Google ADK: Commit-on-Yield

Remember ADK’s yield-based loop? This is why it matters. A SessionService owns the session lifecycle, and state is committed when the agent yields an event — the Runner commits the event’s changes (state_delta) before letting the agent resume. The upside is read-after-commit consistency: when the agent wakes back up, the state it reads is guaranteed to reflect what was just committed. ADK also scopes state with prefixes — user: (shared across that user’s sessions), app: (shared across all users), temp: (discarded after the invocation) — a clean built-in answer to “how long should this piece of state live?”

5.4 LangGraph: Checkpointers and Threads

LangGraph persists a graph’s state as checkpoints, scoped to a thread_id. You attach a checkpointer when you compile the graph (builder.compile(checkpointer=...)); InMemorySaver is the in-memory implementation, with durable backends available. Each conversation is a thread; the checkpointer snapshots state so the thread can be resumed. Compiling with a checkpointer is also what turns on durable execution: a graph can “pause and resume … even after interruptions or failures,” and if a node fails mid-run it restarts from the last successful super-step (a single tick of the graph) without re-running the nodes that already succeeded.

LangGraph then lets you tune when it pays the persistence cost, via three named durability modes — "exit" (write only when the graph finishes; fastest, no intermediate state), "async" (write in the background while the next step runs; a good balance, with a small risk of a lost checkpoint on a mid-step crash), and "sync" (write before each step starts; most durable, some overhead). It’s a clean dial from “cheap and forgetful” to “safe and slower.” For data that needs to outlive a single thread, a separate cross-thread Store holds key-value state — the split LangGraph draws is checkpointers for short-term, thread-scoped memory and stores for long-term, cross-thread memory.

5.5 Temporal: The Agent Is a Durable Workflow

Temporal takes the most radical position: the entire agent is a durable workflow, and durability comes from replay. If the agent crashes mid-run, Temporal reconstructs its state by replaying the recorded event history rather than re-executing the steps that already completed. An agent that crashes after finishing fifteen web searches resumes without repeating them — the completed work is durable, the costs already paid stay paid. Human-in-the-loop pauses use the same machinery: the workflow can wait “minutes, hours, or days” for a person to respond and pick up exactly where it left off. For agents that run for hours and cost real money per step, this crash-resilience is the headline feature.

5.6 AWS AgentCore: Managed Memory Tiers

AgentCore offers managed Memory with both short-term (multi-turn conversation) and long-term (persists across sessions) tiers, and stores can be shared across agents. It’s the “don’t build this yourself” option — you get the durability without owning the storage.

The design question for you: how much does a lost run cost? For a thirty-second task, in-memory state is fine. For a multi-hour research agent making paid API calls, you want Temporal-style replay or checkpointing so a crash doesn’t restart from zero. Match the durability model to the cost of losing progress.

6. Concurrency Within a Single Agent

A single sequential loop is slow when subtasks are independent. The harness’s answer is concurrency — and, helpfully, it reuses the sub-agent machinery we already met for context isolation.

6.1 Orchestrator-Worker

Anthropic’s multi-agent research system uses an orchestrator-worker pattern: a lead agent coordinates and delegates to specialized subagents that run in parallel — spinning up “3–5 subagents in parallel rather than serially.” Each worker explores its slice with its own context and reports back a summary. Because a context window over 200K tokens gets truncated, the lead agent saves its plan to memory so it survives.

This is fan-out/fan-in: split the work, run the pieces concurrently, gather the results. The same pattern gives you both speed (parallel execution) and context economy (each worker’s exploration stays in its own window).

6.2 Deterministic Orchestration

Google’s ADK exposes concurrency as explicit, inspectable control flow rather than something the model decides. Its workflow agents orchestrate sub-agents without consulting an LLM: SequentialAgent runs them in order, ParallelAgent runs them concurrently, and LoopAgent repeats until a condition is met. When you want predictable, auditable orchestration — “always run these three in parallel, then combine” — deterministic workflow agents beat letting the model improvise.

6.3 Isolation at the Infrastructure Level

AWS pushes isolation down to infrastructure: AgentCore Runtime gives each session “true session isolation,” running it in “a dedicated microVM with completely isolated CPU, memory, and filesystem resources.” Concurrent sessions can’t step on each other because they’re in separate micro-VMs, and when a session ends (they can run up to eight hours) “the entire microVM is terminated and memory is sanitized” — no residue leaks to the next tenant. It’s the strongest isolation guarantee in this roundup, and framework-agnostic (it hosts LangGraph, CrewAI, ADK, OpenAI’s SDK, Strands, and more).

6.4 Where Concurrency Breaks

Running agents in parallel introduces failure modes a single loop never has. A 2025 study, “Why Do Multi-Agent LLM Systems Fail?”, built a taxonomy (MAST) of 14 failure modes in 3 categories, and the one most relevant here is inter-agent misalignment — coordination and communication breakdowns: agents resetting the conversation, failing to ask for clarification, derailing the task, or withholding information from each other. The practical warning: when you fan out to parallel sub-agents, the handoff design — what each worker is told, what it returns, how the lead reconciles conflicting results — is exactly where things break. Concurrency buys speed at the cost of coordination risk. Budget for the coordination.

7. Observability

You cannot operate what you cannot see. A multi-step agent that fails on turn 47 of 60 is a nightmare to debug without a record of what it was thinking and doing. Observability is the harness feature that turns “it broke, no idea why” into a trace you can read.

7.1 Traces and Spans

OpenAI’s SDK has the most concrete tracing model of the systems surveyed. Every run is automatically wrapped in a trace, and within it the SDK emits typed spans for each kind of operation:

• agent_span() — an agent’s execution
• generation_span() — an LLM call
• function_span() — a tool call
• guardrail_span() — a guardrail check
• handoff_span() — a handoff to another agent

Each span records timing, a trace_id, and a parent_id, so you can reconstruct the full tree of what called what and how long it took. You can disable tracing, gate capture of sensitive inputs/outputs, and swap the export destination — adding a processor alongside OpenAI’s backend, or replacing it entirely to send traces wherever you like. That parent/child structure is exactly what you need to debug a multi-step run: you can see that turn 47 spent eight seconds in a tool call that returned an error the model then ignored.

7.2 The OpenTelemetry Question

A natural question: is there a standard for agent traces, or does every vendor roll its own? It’s converging — and there’s now something concrete to point at. OpenTelemetry, the incumbent standard for tracing ordinary software, has spun up a dedicated GenAI semantic-conventions effort that defines named agent spans: create_agent, invoke_agent, plan, execute_tool, invoke_workflow. That’s exactly the vocabulary you’d want to describe an agent run in a vendor-neutral way.

The honest caveat is maturity: as of this writing, every one of those GenAI-specific spans and attributes is marked “Development” status — which in OpenTelemetry’s own terms means breaking changes are allowed and the spec could still be reworked. So the convention exists and is worth tracking, but it hasn’t stabilized, and which frameworks actually emit these spans in practice is still thin. On the vendor side, AWS’s AgentCore Observability emits telemetry in “standardized OpenTelemetry (OTEL)-compatible format” with per-step visualizations, and OpenAI’s tracing is proprietary but exportable. If vendor lock-in on observability worries you, favor systems that speak OTEL — just don’t expect the agent-span schema to be frozen yet.

7.3 Logs as Hooks

Anthropic’s approach is less about a tracing standard and more about interception points. The Agent SDK exposes lifecycle hooks — PreToolUse, PostToolUse, Stop, SessionStart, SessionEnd, UserPromptSubmit, and more — as callbacks that can “validate, log, block, or transform” behavior. Want to log every file write, or block a dangerous command before it runs? Register a PreToolUse hook. For attributing work in concurrent runs, messages from a subagent carry a parent_tool_use_id so you can tell which subagent did what. Hooks are observability and control: the same mechanism that logs an action can veto it.

7.4 Durability as Observability

A nice two-for-one: Temporal’s event history is both its recovery mechanism and its audit trail. The Temporal UI shows “every agent invocation, every search Activity running in parallel, the full event history.” The same log that lets the agent resume after a crash lets you replay exactly what it did. When durability and observability share a substrate, you get both for the price of one.

8. Putting It Together — A Cross-System Comparison

We’ve covered seven concerns. Here’s how the major systems answer each, side by side:

Two axes organize this table. One runs from self-hosted to managed: you write the loop (Anthropic Client SDK, LangGraph) at one end, the cloud runs it (AgentCore Harness, OpenAI server-side sessions) at the other. The other runs from reactive to deliberative: ReAct-style loops (Anthropic, OpenAI) versus explicit graphs (LangGraph, ADK workflow agents) versus durable workflows (Temporal).

There’s no winner. There’s a fit. A quick research script wants a reactive in-process loop with throwaway state. A multi-hour agent making paid API calls wants Temporal-style durability. A regulated enterprise deployment wants AgentCore’s micro-VM isolation and OTEL observability. The skill is matching the harness to the job — and you can only do that once you know what each piece is for, which is what this chapter was about.

Summary: What to Internalize

• The model is not the agent. The harness — the loop, tool dispatch, context management, state, concurrency, observability — is where production reliability lives.
• The loop is ReAct, everywhere. Thought → Action → Observation, until done. The observation, grounding the agent in real feedback, is the whole point.
• The big design choice is reactive vs. deliberative. Reactive adapts but can get stuck; deliberative stays on-task but is rigid. Most real systems blend them.
• Context management is the active frontier. Compaction, context editing, and sub-agent isolation all serve one goal: keep the active context small. If your agent dies on long tasks, look here first.
• Stall handling is blunt today. Hard turn limits, typed errors, and self-verification checkpoints. Start with a turn cap.
• State durability is a real choice. Match it to the cost of losing a run — from in-memory to checkpointed to fully replayable.
• Concurrency reuses sub-agents for both speed and context economy — but the handoff design is where multi-agent systems break.
• Observability turns failures into traces. Spans, hooks, and event histories are how you debug an agent at 2 a.m. Favor OTEL if lock-in worries you.