Every production agent we have run has failed in one of six ways. Not sixty — six. The failure modes repeat so reliably that we now wire a specific guardrail for each one before an agent touches real work, and those guardrails catch roughly 19 of every 20 incidents before a human ever sees them.
This list comes from 14 months of running agents on our own pipeline and on client systems. Thousands of runs, a few dozen incidents worth writing down, six patterns.
Here are the six, the guardrail that catches each, and roughly what each guardrail costs to build.
Failure one: the confident wrong answer
The agent states a false thing fluently — an invented statistic, a wrong price, a feature the product does not have. This is the failure everyone fears and the one vague prompting advice does not fix. The guardrail is a claim-check pass: a second model extracts every factual claim from the output and verifies each against the source material the agent was given. Unverifiable claim, blocked output.
Ours flags about one output in 40. Most flags are small — a rounded number presented as exact, a date off by a year. Small is exactly what humans miss on review.
A useful side effect: the flag log becomes a curriculum. Every caught claim tells you which source the agent lacked, and feeding those sources back into the brief cut our flag rate by a third in two months.
Cost: about three hours to build and roughly two cents per run. It is the most expensive guardrail on this list and the first one we install anyway.
Failure two: the runaway loop
The agent hits an error, retries, hits it again, and spirals — burning tokens all night on a task that failed at step two. The guardrail is hard budgets: a step cap, a spend cap, and a wall-clock cap on every run. Ours are 24 steps, $1.50, and ten minutes. Trip any one and the run halts, logs its state, and files a ticket instead of trying harder.
We learned this the standard way. Before caps, an agent retried a broken fetch 412 times overnight. The bill was only $19, but the same loop with a bigger model and a webhook would not have been.
Set the caps embarrassingly low at first. An agent that legitimately needs more than 24 steps will tell you so by tripping the cap, and raising a budget is a one-line change you make with evidence in hand.
Cost: about an hour. This is the highest ratio of protection to effort you will find anywhere in agent work.
Failure three: silent drift
Nothing breaks, nothing errors — the output just gets a little worse every week until a client says something. Causes include model version changes, a prompt edit that helped one case and hurt five, and upstream sources changing shape. The guardrail is a golden set: 20 frozen inputs with human-approved outputs, re-run and scored every Sunday night. A score drop of more than five points pages a human.
Drift is the failure mode that killed more client trust than any other in our first year, precisely because no alarm goes off on its own.
Score with a rubric, not vibes: we grade each golden output on accuracy, structure, and tone, zero to 100, and chart the trend. The line matters more than any single week's number.
Cost: half a day to build the set and the scorer. The discipline is refreshing the set quarterly so it still resembles real work.
Failure four: scope creep
The agent does its job and then a little more — rewrites the pricing paragraph while fixing a typo, touches the nav while updating a post. The guardrail is a write allowlist plus a diff cap. Each agent declares the paths and collections it may modify, and any diff over 300 lines or outside the allowlist queues for human approval instead of shipping.
The allowlist matters more than the cap. An agent that can only write to the blog collection can have a very bad day and still not hurt you where you live.
Diff caps carry a second benefit nobody advertises: they keep changes reviewable. A human can actually read a 300-line diff. Nobody reads a 3,000-line one, and unreviewable is another word for unguarded.
Cost: about two hours, mostly deciding what the allowlist should say. That conversation is worth having even if you never build the agent.
Failure five: the poisoned handoff
One agent's slightly-wrong output becomes the next agent's input, and the error compounds through the pipeline until the final artifact is confidently absurd. The guardrail is schema validation at every seam: each handoff between steps is a typed contract, and every payload is validated against it before the next step runs. Bad output gets rejected at the seam it came from, not three steps downstream where the cause is invisible.
The rule we follow: if you cannot write the schema for a handoff, the two steps are not ready to be separate agents.
It sounds bureaucratic until you watch it work. The week we added seam validation, time-to-diagnose for pipeline failures dropped from hours of log spelunking to the name of the seam in the alert.
Cost: about two hours per seam. Our content pipeline has four seams. The validators have rejected 61 payloads this year, each one a debugging session we did not have.
Failure six: acting on stale context
The agent answers from data that was true last month — an old price, a retired service, a stat from the previous quarter. The guardrail is a freshness TTL on retrieved facts. Every fact in our retrieval layer carries a fetched-at timestamp, and anything past its time-to-live — seven days for stats, 24 hours for prices — gets refetched or flagged before the agent may use it.
This one is invisible until it is public. A stale price quoted to a prospect costs more than every token you saved by caching.
The TTLs live in one config file, reviewed monthly. When a client's business changes speed — a pricing revision cycle, a product launch — the file changes with it the same day.
Cost: half a day if your retrieval layer already stores metadata. Longer if it does not, which is itself a finding.
Build the guardrails before the agent
Add it up: roughly two working days of guardrail construction covers all six failure modes. That is less than the cleanup bill for one uncaught incident, and far less than the client trust an incident spends. In our codebase, guardrail code is about 40% of total agent code, and we consider that ratio healthy rather than embarrassing.
The order we install them: budgets first, allowlist second, schema seams third, claim-check fourth, golden set fifth, TTL sixth. Cheapest and broadest protection first.
None of this makes agents safe. It makes their failures small, visible, and cheap — which is the only kind of safe that exists in production.
If you want the list applied to your own pipeline, this audit is the first week of every Avakata agentic engagement.