The demo works. Production doesn't.

What actually changed

Not the model. Not the prompt.

The harness

The system around the model call.

Almost every agent failure lives in one of these five, not in the prompt.

When does this stop?

• A step budget, not vibes. Max iterations, max tool calls, max wall-clock time.
• A termination condition checkable by code, not the model announcing “done.”
• A defined escape hatch: what happens on the final attempt when it still has not worked.

What does the model see right now?

• Context is assembled per step, not accumulated forever.
• Belongs: current state, retrieved facts, the last relevant results.
• Does not: the entire conversation history by default.
• Stale context is worse than missing context: it looks authoritative, and the model cannot tell it is old.

What can it touch?

• Allowlist per step, not per agent.
• Read and write are separate privileges. Most steps only need read.
• Anything destructive, external, or irreversible gets a gate or a dry run.
• The blast radius of a wrong decision is a design parameter, not an accident.

How do I know this output is good?

Three tiers, cheapest first:

What survives?

• What has been attempted, and what failed, and why
• What the user has already confirmed or rejected
• Where we are in the workflow

How they interact

The layers are not independent. This is where most designs go wrong.

• Evaluation without state → retries that fail identically
• Context without workflow control → context grows until quality degrades
• Permissions without workflow control → an unbounded loop with write access
• State without evaluation → you faithfully persist a wrong answer and build on it

Where everyone starts

def agent(user_input):
    result = model(user_input)
    return result

Failure modes hiding in one line:

Five failures. Five layers. That is not a coincidence.

Move the guarantee into code

from pydantic import BaseModel

class Result(BaseModel):
    action: str
    target_id: str
    confidence: float

raw = model(user_input)
parsed = Result.model_validate_json(raw)   # throws on malformed

Why attempt two can succeed

for attempt in range(MAX_ATTEMPTS):
    raw = model(build_context(state))
    try:
        parsed = Result.model_validate_json(raw)
        state = state.update(parsed)
        break
    except ValidationError as e:
        state.log_failure(str(e))          # error goes back in
else:
    return escalate(state)

Allowlists are per step

ALLOWED = {"search", "read_record"}        # note: no writes

result = model(
    context=build_context(state),
    tools=[t for t in TOOLS if t.name in ALLOWED],
)

if state.steps > MAX_STEPS or state.is_complete():
    return finalize(state)

Side by side

def agent(user_input):
    return model(user_input)

Order matters: schema first (nearly free), then rules, then anything that needs another model call.

Harness or prompt?

Apply it

This is why failing prompts keep growing: each incident adds a sentence, the sentences compete, and past a point they degrade each other. The harness scales the other way: it is code, and code composes.

Map your failure

What we did not cover

• Evaluation sets and regression testing, so a change does not break step four
• Multi-agent handoff, where state becomes the hard part
• Human-in-the-loop gates and approval flows
• Observability: tracing a failure back to the step that caused it
• Cost and latency budgets as a harness concern