Most large companies have tried an AI agent by now, and very few have one that pays off. In McKinsey's April 2, 2026 analysis of agentic AI at scale, nearly two-thirds of enterprises had experimented with agents, but fewer than 10 percent had scaled them to deliver tangible value.
AI agents stall in production because of what is missing around the model, not because of the model. The missing piece is the operational layer: the tracing, evaluations, guardrails, and drift monitoring that make an agent's behavior visible, testable, and bounded. It is the part most projects underbuild.
What a demo skips that production exposes
A demo shows the happy path, and production is everything the demo left out. In production, an agent that calls tools can produce different results from identical inputs. It can loop on a task until it burns through a budget. It can drift off-policy while still looking busy. Worst, it can return an answer that is fluent, confident, and wrong. That is the hardest failure to catch, because nothing about it looks like a failure.
None of these are model defects. They are properties of running a non-deterministic system inside an organization that expects deterministic software. Swapping models does not fix them. Building the operational layer does.
The four parts of the operational layer most agent projects lack
Stalled agent projects usually lack four things, in rough order of how often we find them missing.
Tracing. Tracing records every step an agent takes (each model and tool call, with its tokens and cost) so that any run can be replayed. You do not have to invent the format, because OpenTelemetry publishes semantic conventions for generative AI that define standard spans, metrics, and events for model clients. If you cannot reconstruct why an agent did what it did on a specific run, you cannot debug it. Most stalled projects are stuck right here: the agent is live, it misbehaves now and then, and nobody can see inside it.
Evaluations. An evaluation set is a graded collection of representative cases that runs automatically on every change to a prompt, a model version, or a tool. It is the regression suite for a probabilistic system. "It worked when I tried it" proves nothing when the same input can produce different outputs. A held-out evaluation set is the only way to know whether a change made the agent better or quietly worse.
Guardrails and bounded autonomy. Hard ceilings on spend and iterations make a looping agent fail safe instead of failing expensive. Enforce those ceilings in the workflow orchestration layer, not in a prompt, so they hold when the model does not cooperate. Put a human approval gate in front of anything irreversible. Autonomy is a dial, not a switch, and teams that ship set it on purpose.
Drift detection. Model providers update their models, and your data changes. Both move an agent's behavior without anyone touching the code. Uptime monitoring tells you the service is running. It says nothing about whether the outputs are still good, so output quality needs its own tracked signal.
The second pass we build for consequential tasks
For any task with real consequences, we build a second step that checks the first. One step generates the output. A separate step critiques it against explicit criteria. When the two disagree, the case goes to a person instead of shipping. This review loop is not exotic, and it adds cost. It is still the most reliable defense we have found against plausible-but-wrong output, because the critic does not reuse the reasoning that produced the answer.
Reliability in an agent system is a property of the architecture. You design it in up front, or you retrofit it after the first incident, and the retrofit always costs more. An agent that sits on inconsistent data gives this layer more to catch, which is why we fix the foundation before bolting on the agent.
How should you budget an AI agent project?
Budget for the operational layer as a named line item, because it is most of the cost of a reliable agent and almost none of the cost of a demo. Tracing, the evaluation harness, guardrails, and monitoring are where the engineering goes. A proposal that is all model and no operational layer is a proposal for a demo, and the McKinsey scaling gap is what a pile of those demos looks like. An agent that began as someone else's prototype also needs the rebuild described in taking an inherited AI proof of concept to production.
The same layer pays a governance dividend. Article 12 of the EU AI Act, Regulation (EU) 2024/1689, requires high-risk AI systems to allow the automatic recording of events over the lifetime of the system. The traces and evaluation records you build for reliability are largely that evidence, produced as a byproduct of running the system rather than as a second project.
Everyone has access to strong models in 2026. The teams getting a return on agents treated observability, evaluation, and bounded autonomy as the foundation, not the finish. Decide which kind of project you are funding before the contract is signed. If you want help drawing that line, our AI implementation framework is built for it. AI that ships, not AI that demos.