Half of agent pilots failing may be normal. Not measuring them isn't.

Vendors have been telling buyers that the pilot phase of AI agents is over. "The era of the pilot is over. The era of the agent is here," Google Cloud chief executive Thomas Kurian said at Google Cloud Next in April. A new Linux Foundation report quotes the line, then spends most of its pages on the evidence that makes it hard to believe.

The report, "Enterprise Transitions to the Agentic Era", was published in September. It draws on 25 interviews with practitioners who volunteered from Linux Foundation communities. They include engineers, architects, product leaders and executives. Two of the three authors work at Guru Technologies and Futurewei, which are also among the report's sponsors. Its central finding is that model capability is no longer the main constraint. Trust, governance and cost now set the pace.

This is a qualitative study. It tells you what experienced people are seeing, not what share of companies are seeing it. Read it that way and it is one of the more useful documents on agents this year, because the people in it are the ones who run the systems after the demo.

Failure is expected. Unmeasured failure is the problem.

The report takes on the familiar claim that most AI pilots fail. The interviewees mostly did not treat that as alarming. "50 percent of pilots failing is probably normal," said Jonathan Rochelle. "And if it's not, then you're not experimenting enough."

The same interviewees were firm about what that experimentation should cost you in discipline. Each pilot, in Rochelle's words, should impose "measurement": a stated goal and a record of whether it was met.

Diego Gosmar put the common failure plainly: "most failures aren't model failures, they're measurement and integration failures; pilots that automated a demo rather than a process." Xavier Shay, chief executive of Up Bank, described where companies get stuck: "The barrier to building an impressive prototype is extremely low, but scaling to a repeatable, production-grade workflow you can trust is where organizations stall."

For a 100-person distributor or a 300-person services firm, the difference between a demo and a process is concrete. A demo shows the agent drafting a quote. A process includes the pricing exceptions, the customer who emails a photo instead of a part number, the handoff to whoever approves discounts and the record that ends up in your ERP. An agent that handles the first and not the rest has not saved anyone time yet.

What to measure instead

The report's recommendation on return on investment is the part to keep. It advises defining "a completion- or resolution-rate metric and a fully loaded cost-per-outcome measure against the human-process baseline," and warns against using tokens consumed or number of agents deployed as stand-ins for value.

Put plainly, a pilot needs answers to three questions before it starts:

1. What counts as done? An invoice matched and posted. A support ticket closed without being reopened. A purchase order issued with the right terms.
2. What does it cost today to get that done? Staff time at a loaded rate, plus the error and rework cost you already carry. This is your baseline.
3. What does it cost with the agent? Model and software fees, integration and maintenance time, and the time your people spend reviewing and correcting agent output.

Token counts and agent headcount are easy to report and say nothing about whether the job got cheaper. A separate paper on coding agents, posted to arXiv on September 24, reaches the same conclusion from the cost side. It recommends reporting cost per verified task rather than tokens per user.

Accountability does not move to the agent

Interviewees were consistent that responsibility stays with the company deploying the agent. Angie Jones, who helped build Block's Goose agent, described her team's rule that "a human had to be held accountable." Even when an agent opened a pull request, "your byline is still attached, so you're accountable for it." Arnaud Giuliani, co-founder of Kotzilla, described his team's rule that "each execution is always human validated and never pushed directly to production."

That has a direct cost implication, and it is the one mid-market budgets most often miss. Review time is part of the cost per outcome. If your controller has to check every agent-matched invoice, the saving is the difference between matching and checking, not the whole matching task. Some interviewees working inside single companies named "the capacity to review agent output" as a bottleneck in its own right.

A gateway is treated as a baseline

Several interviewees described putting a gateway, a single control point, between their agents and the models and tools those agents use. The report says they treated it "as a baseline for production, not a later optimization." A gateway gives one place to log what agents do, enforce cost limits and switch models.

A company with 60 employees probably will not run its own. But the question translates directly into buying criteria. Can you see what the agent did, set a spending cap and change the underlying model without rebuilding the workflow? Many companies cannot yet. In a separate VentureBeat survey of 101 organizations with at least 100 employees, 27% said they had no real-time way to stop runaway agent costs and relied on reactive monitoring. That survey was self-selected, so treat the figure as a signal rather than a benchmark.

Shadow AI, and tools nobody remembers

The report opened with a hypothesis that companies are in a "shadow AI" phase, where employees experiment before formal adoption. Interviewees supported it. Christian Maitre described the end state as "shadow amnesia: tools built, deployed, forgotten, … still connected, but outside any governance perimeter."

That will sound familiar to anyone who has found an old automation still emailing a customer list to a former employee. Agents make the problem more serious because they can act, not just report. A simple inventory helps: what agents and AI automations exist, who owns each one, what systems each can touch and when each was last reviewed. It does not need to be elaborate.

Not every request needs the most expensive model

The report points toward a portfolio of models. Small or locally hosted models would handle high-volume routine work, and frontier models would take the hard cases. One interviewee estimated that 80% to 90% of requests by volume could run on small or local models. That is one person's estimate, not a finding. The report also notes that running models in-house is still hard, and one interviewee said local models are "not yet ready just to run on [any] laptop."

For most mid-market companies, the practical version is simpler than running your own models. Ask whether the tools you buy let you choose a cheaper model for routine work.

The dissent worth hearing

Not everyone interviewed believes this is moving fast. "Agent adoption will also take a decade … the slowest move is not technology, it's human change management," said Guillaume Blaquière of Carrefour, who also wrote the report's foreword.

The report also notes a split on the biggest bottleneck. Trust and governance came up most often across all the interviews. People running AI inside a single company more often named cost, data quality, organizational change or the capacity to review agent output. That second list is the mid-market operator's list. It is also a more honest description of the work than any keynote.

End the unmeasured pilot

The pilot era is not over for most companies. What should end is the unmeasured pilot. Before the next agent project starts, write down what counts as done, what it costs today, who reviews the output and how you will turn it off. A pilot that fails against that standard has still told you something useful. A pilot measured only in enthusiasm has not.

Sources

• Curran, H., Carter, H., Lai, A. "Enterprise Transitions to the Agentic Era." Linux Foundation Research, September 2026. DOI 10.70828/QFRW8479. https://www.linuxfoundation.org/hubfs/Research%20Reports/lfr_26_enterprise_transition_ai_era_091826a.pdf?hsLang=en
• Abbasi, A., Aqrawi, A., Kwartler, T. "Control the Harness, Control the Cost." arXiv 2609.28919, 24 September 2026. https://arxiv.org/abs/2609.28919
• VentureBeat Pulse. "Agentic orchestration: enterprise AI organizations have a deployment problem, not a platform problem." July 23, 2026. https://venturebeat.com/resources/agentic-orchestration-enterprise-ai-organizations-have-a-deployment-problem-not-a-platform-problem-and-most-are-calling-chatbots-agents