The cost of a wrong assumption: how we trace agent errors back to their root

Key takeaways

• 73% of production errors we have traced in the past six months had their root cause at or before step two of a multi-step pipeline — not in the final step where the error became visible.
• The three most common root causes: stale inputs (the data the agent read at step one was no longer accurate by the time the action ran at step five), ambiguous task scope (the prompt allowed two valid interpretations and the agent chose the wrong one), and missing guard conditions (a condition that should have triggered a human-in-the-loop check was not defined).
• Agent error tracing requires a decision log, not just an output log: you need to know what assumption the agent made at each step, not just what it produced at the end.
• Every traced root cause should resolve into one of three actions: update the input data pipeline, tighten the prompt, or add a guard condition. If none of these apply, the error is out of scope and belongs in the escalation category.

The wrong place to start debugging

Your content agent published a piece with a statistic that was wrong. The number had been revised two months earlier. You stare at the final output — the published article — and try to figure out what happened. You check the prompt. You check the output format. You check the model version. Nothing obvious.

This is the most common debugging failure in agentic systems: teams examine the final output when the error was introduced three steps upstream. The output is just where the problem became visible. The decision that caused it happened much earlier, and without a record of that decision, you are guessing.

What a decision log captures

A decision log is not an output log. An output log records what the agent produced at each step. A decision log records what the agent believed at each step — and why it acted the way it did.

At each step, a decision log captures five things:

• Input: what data or context the agent received
• Assumption: what the agent treated as true about that input
• Action: what the agent did based on that assumption
• Output: what it produced
• Confidence: how certain the agent was, and on what basis

The assumption field is the critical one. Outputs are observable. Assumptions are not — unless you log them explicitly. An agent that receives a statistic and treats it as current is making an assumption. If that assumption is wrong, every downstream step inherits the error. The output at step 7 looks plausible. The mistake was at step 1.

Tracing a real failure

A content agent published an article citing a market size figure of $4.2 billion. The correct figure, as of two months prior, was $6.1 billion — the original source had issued a revision. The published article was factually wrong.

Here is what the decision log showed:

Step 1 — Source retrieval. The agent crawled a research aggregator and retrieved the $4.2B figure. The page had not been updated since the revision. The agent logged the value as retrieved. It did not log when the source was last updated. Assumption: this figure is current.

Steps 2–6 — Content planning, outline generation, section drafting. The $4.2B figure passed through each step as a fact. No step questioned its freshness. Each step's assumption was inherited from step 1: this figure is current.

Step 7 — Final draft assembly. The agent cited the figure in the opening paragraph. Confidence: high. The number had appeared consistently across six prior steps without challenge.

Root cause: stale input at step 1, with no freshness check. The agent had no mechanism to ask when was this data last verified?

Fix: add a last-crawled timestamp check to the source retrieval step. If the source was crawled more than 30 days ago, flag it for human review before passing the value downstream. One guard condition at step 1 would have caught this before step 2.

The three root causes

1. Stale inputs

The example above is the canonical case. But stale inputs appear in subtler forms too. A product description agent pulls specs from an internal knowledge base that was last synced six weeks ago. A pricing agent references a rate card that was superseded by a new contract. In both cases, the agent's assumption — this input reflects current reality — is wrong, and nothing in the pipeline challenges it.

The fix is the same in every case: attach a freshness assertion to every external input. Log the source timestamp alongside the value. Define a staleness threshold. Fail loudly when the threshold is exceeded rather than passing a potentially wrong value downstream.

2. Ambiguous task scope

A summarization agent was given the instruction: summarize the key findings for the executive audience. The prompt did not specify whether "key findings" meant the three highest-confidence findings or the three most commercially significant ones. Both interpretations are valid. The agent chose the highest-confidence findings. The stakeholder wanted commercial significance.

The decision log showed the agent's assumption at step 1: key findings = highest confidence score. That assumption was never stated in the prompt, and no one had defined it. The output was technically correct under one interpretation and wrong under the one that mattered.

Fix: treat ambiguous scope as a guard condition. When a prompt contains terms that admit multiple valid interpretations — key, important, relevant, appropriate — the agent should log the interpretation it chose and, where the stakes are high, surface it for confirmation before proceeding.

3. Missing guard conditions

A publishing agent was designed to route low-confidence outputs to a human reviewer before publication. The routing logic was defined for confidence scores below 0.6. The agent published a piece with a confidence score of 0.61 — just above the threshold — that contained a factual error a human reviewer would have caught in thirty seconds.

The decision log showed the agent correctly evaluated its confidence as 0.61 and correctly determined that 0.61 ≥ 0.6. It followed its rules exactly. The problem was that the rules were incomplete. No one had defined what to do when confidence was borderline — above the threshold but not comfortably so.

Fix: guard conditions need ranges, not just thresholds. A confidence band of 0.6–0.75 should trigger a soft review flag, not a binary pass. The agent's decision log should record which band it fell into, not just whether it cleared the threshold.

Why output logs are not enough

An output log tells you what the agent produced. It does not tell you what the agent believed when it produced it. The distinction matters because an agent can produce a plausible-looking output based on a completely wrong assumption.

Think of it like a navigation error. If you are driving and take a wrong turn at mile 3, your GPS will show you arriving at a location that looks like a valid destination — it just is not the one you intended. Looking at where you ended up does not tell you where you went wrong. You have to replay the route from the beginning.

A decision log is the replay. Without it, you are debugging by examining the destination and guessing about the route.

The two-step debugging rule

Before you look at the final output, open the decision log and read it from step 1. Find the first assumption that was wrong. That is your root cause.

Everything after that assumption is a consequence. Fixing the output — patching the published article, correcting the number, rerunning the final step — addresses the symptom. Fixing the assumption at step 1 addresses the failure. The two-step rule keeps you from spending an hour debugging step 7 when the answer is at step 1.

Frequently asked questions

How do you debug an AI agent that produced incorrect output?

Start from step 1 of the decision log, not the final output — the error almost never originates where it surfaces. Trace forward until you find the first assumption that diverges from ground truth; that is your actual failure point. Map it to one of three root cause categories: stale input, ambiguous task scope, or a missing guard condition. Once categorized, the fix is mechanical — refresh the input, tighten the scope definition, or add the missing guard. Chasing the output directly wastes time and usually treats a symptom.

What is a decision log in an agent pipeline?

A decision log is a structured record that captures, at each pipeline step, the input received, the assumption made, the action taken, the output produced, and the agent's confidence level. It is distinct from a plain output log, which records only what the agent returned. The assumption field is what makes it useful for debugging: it exposes the internal reasoning the agent used to move from input to action, which is exactly where most errors originate. Without the assumption field, you can see that something went wrong but not why.

What are the most common root causes of agent errors?

The three root causes are stale inputs, ambiguous task scope, and missing guard conditions. Stale input means the agent acted on data that was no longer accurate — for example, a pricing lookup cached from the previous day. Ambiguous scope means the task definition left enough room for the agent to make a plausible but wrong interpretation. A missing guard means no condition existed to catch an out-of-range value or an unexpected state before the agent committed to an action. Across traced pipelines, 73% of errors had their root cause at or before step two of the decision log.