Why AI Agent Projects Fail: 4 Preventable Patterns

Two years ago, the honest conversation about AI agents involved a lot of “some of them work, some don’t, hard to predict which.” That conversation is over.

The survey data points the same way:

• Anthropic’s December 2025 survey of 500+ technical leaders: “80% of organizations report their AI agent investments are already delivering measurable economic returns.”
• PagerDuty’s agentic AI survey (April 2025, 1,000 IT and business executives): companies expect an average return of 171% on their agentic AI investment. That is an expectation, not ROI measured across deployments.

The AI agent market has become one of the safer bets in enterprise software. Which means the interesting story is no longer “does it work?” — it is “why does it not work when it doesn’t?”

The projects that fail to show ROI are worth dissecting because the failure patterns are boringly consistent. Four of them account for almost every failed deployment we have seen.

What the data says, and why success is now boring

The Anthropic survey does not say the other organizations failed, and it does not put a time limit on the returns. What it does say is that measurable returns are now the norm, so a rational SMB should be asking not “should I deploy?” but “which workflow first, and how do I avoid the four patterns below?”

Failure #1: No eval suite

What it looks like: the agent is built, tested manually against 10-20 examples, and shipped. Six weeks in, the team notices the outputs have drifted (a model update, a data change, a prompt edit) but there is no systematic way to measure how bad the drift is. Trust collapses. Adoption stops. The agent gets quietly retired.

Why it happens: eval suites feel like engineering overhead in a project that was sold as “just plug in AI.” Nobody budgets for the 20-30 test cases that would have caught the drift.

The fix: every agent build should ship with a documented eval suite of at least 20 test cases covering the top failure modes for that workflow. Re-run monthly, or on any model/prompt change. If the eval regresses noticeably, that is a triage event, not a “we’ll look at it next quarter” event.

Our internal rule: no agent goes live without an eval suite, and no agent renews its retainer without a passing monthly eval. This one rule alone eliminates a large share of failures.

Failure #2: Scope too broad

What it looks like: the initial spec says the agent will “handle customer support.” Six months later, the agent kind-of-handles some of customer support, gets some tickets right and fumbles the ambiguous ones, and the team defaults to routing everything to humans anyway because they cannot predict which tickets the agent will handle. The agent produces no measurable time savings because trust never crossed the threshold where humans would delegate confidently.

Why it happens: everyone overestimates the model’s ability to handle edge cases in the demo, and underestimates how often the edge cases occur in production.

The fix: narrow scope aggressively at the spec stage. Instead of “handle customer support,” pick “auto-draft the first response for Tier-1 tickets in the shipping category, sent for human review before send.” That version can be evaluated, adopted, and iterated. The broader version cannot.

The Workforce Starter engagement is designed around this rule: five agents, each with one narrow well-defined job. It is not the exciting shape — it is the shape that ships.

Failure #3: No sponsor, no adoption plan

What it looks like: the agent works technically. It sits in a dashboard nobody logs into. Six months in, at review time, no one can point to hours saved because no one used it consistently. The renewal conversation dies.

Why it happens: the org treated the agent like an IT project (build → hand off → done). Adoption is a change-management problem that IT projects rarely resource.

The fix: every agent build needs (a) a named executive sponsor with skin in the game, (b) a named team lead who will own daily use, (c) a 30-day adoption plan that includes at minimum: a kickoff walkthrough, a Week-2 check-in, a Week-4 KPI review. The Care retainer is priced to include this scaffolding because deployments without it fail.

In our post-mortems, this is the #1 predictor of failure. Agents that ship without a named team lead are often unused months later; agents with an active team lead and a scheduled Week-4 review get used.

Failure #4: KPI defined after the fact

What it looks like: at renewal, someone asks “how much time did this save us?” and nobody knows, because the baseline was never measured and the “after” number was never defined. Defensive claims get made (“we all feel like it saved us time”), the CFO is unmoved, and the retainer gets cut in the next budget cycle.

Why it happens: measuring baselines feels like slowdown when everyone is eager to ship. So the KPI conversation gets skipped, and then it is too late to reconstruct.

The fix: define the KPI in the spec, measure the baseline in the same week you sign the SOW, measure the “after” number monthly using the same method. Ugly baselines are fine — the point is to have a before number that survives contact with the after number.

Practical KPIs that work: hours saved per week (measured via self-report tally, 5 minutes/week), error rate (measured via QA sample of 20 outputs/month), response time (measured via system logs), throughput (measured via ticket/document count). Pick one primary KPI, measure it monthly, defend it at renewal.

A 30-minute pre-mortem for your next agent

Before you sign the SOW on your next agent build, spend 30 minutes running through these ten questions. If you cannot answer confidently on any three, delay the build.

1. Which specific workflow will this agent do? Describe it in one sentence with a verb.
2. What is the current time cost of that workflow per week? (If unknown, measure for one week before signing.)
3. What does correct output look like? (Show me 5 examples.)
4. What does incorrect output look like? (Show me 5 examples of things the agent should not do.)
5. Who is the executive sponsor? (Name a human.)
6. Who is the team lead who will own daily use? (Name a different human.)
7. What is the primary KPI, in one number?
8. How will the baseline be measured?
9. Who will re-run the eval suite each month?
10. What is the plan if the agent underperforms in the Week-4 review?

Ten answers, 30 minutes. This is the cheapest insurance policy against a failed project.

How we run our own builds against these four

Because the failure patterns are well-known, so is the discipline that prevents them. Our own build methodology has one dedicated step for each of the four:

Failure mode	Our mitigation
No eval suite	Every build ships with a documented 20-test-case eval suite. Retainer includes monthly re-run and delta report.
Scope too broad	Discovery call is oriented around the “one narrow workflow” principle. We routinely refuse builds where the spec cannot be narrowed.
No sponsor / adoption plan	Sponsor + team lead named in the SOW as required fields. Care retainer includes the 30-day adoption scaffolding.
KPI defined after the fact	Primary KPI + baseline measurement is written into the spec before build begins.

We are not claiming this makes our builds bulletproof. What we are claiming is that these four disciplines shift the odds considerably, and every SMB that hires any agency should verify their finalist works this way.

From pilot to programme: why the second agent starts from zero

There is a fifth pattern, and it only shows up after a success. The first agent works. The company asks for a second one in another department, and the second build costs almost as much time as the first. Nothing carried over.

The reason is that the first agent was built as a project, not as the start of a programme. Its knowledge lives in its own prompt and its own documents. Its rules were agreed in a meeting with one department head. Its tests cover only its own task. The second agent has to rediscover all three.

Three choices made during the first build prevent this:

• Write the knowledge into a shared company brain, not into the agent. The second agent reads the same files and only adds what is new. See the AI Company Brain.
• Write the rules as company rules. Autonomy levels, approval rules and owners follow one template, so the second department fills in a form instead of starting a debate.
• Run every agent in one environment, with one sign-in, one log and one budget, instead of a new setup per pilot.

If you are planning more than two agents, plan them as a programme from the first one. Our AI transformation roadmap shows the four phases, and AI transformation services shows how we run them for mid-size and large companies.

---

The AI agent category has become boring in the best way. Most deployments succeed. The failures happen for known reasons that a 30-minute pre-mortem eliminates.

If you want to pressure-test a specific agent idea against the four failure modes above, the 30-minute discovery call is the fastest way to do it.

---

Key terms in this post: AI agent · eval suite · Workforce Starter · retainer · KPI