We Deployed 5 AI Agents for Accounting Firms: Here's What Broke (and What Worked)

This is the field-note version of the accounting vertical pack, the same engagement we describe on the accounting industry page, but with the parts that go wrong, not just the parts that go right.

A note on transparency: across the first three deployments, two firms have given explicit permission to share aggregate findings. Specific firm names and identifying details are redacted; numbers are real but rounded.

The five agents (quick recap)

Before the field notes, the five-agent shape, see the industry page for the full spec.

#	Agent	Tier	What it does
1	Invoice & Receipt Processor	Task	Reads PDFs from email/Drive, extracts vendor/amount/category, drafts journal entries
2	Client Onboarding Concierge	Operational	Sends document requests, runs basic KYC, drafts engagement letters
3	Deadline & Filing Tracker	Operational	Maintains firm-wide tax/statutory calendar, sends client reminders, escalates risks
4	Ledger QA Reviewer	Task	Scans ledger for anomalies, reconciliation gaps, duplicates, daily exception report
5	Monthly Reporting Analyst	Executive	Pulls period close, runs variance analysis, drafts commentary in firm’s voice

This was the design heading into firm #1. The pack survived contact with three real firms with only minor changes, but those minor changes mattered.

What worked first try

Three things landed on day one without needing tuning:

1. The Invoice Processor (Agent #1) at firm #1. The firm had 1,400 monthly invoices across 80 clients. The agent hit 96% extraction accuracy after the standard 5-day discovery phase, and partners signed off on autonomous-mode after the 14-day human-in-the-loop window without modification. The team’s prediction was “this will save 8 hours a week”; actual savings in month two were 22 hours a week.

2. The Deadline Tracker (Agent #3). This one we expected to be the easiest, and it was. Calendar-driven workflows are bounded and well-specified. Once we ingested the firm’s existing tax-deadline matrix, the agent ran clean. Across all three firms, the deadline tracker has not been the subject of a single ticket in production.

3. Eval suite discipline saved us at firm #2. Firm #2 had a regional dialect of accounting terminology, some non-standard journal categories, custom client codes. Our eval suite (20 test cases per agent, built during discovery) caught seven mismatches before launch. Two were trivial; five were “this agent would have made a real mess in production” findings. We fixed them in pre-launch instead of in front of a paying client. The methodology paid for itself in that single moment.

What broke in week 2

Week 2 is the classic “honeymoon ends” point. Three things broke across the first three firms.

1. Onboarding Concierge (Agent #2) sent the wrong document request template. At firm #1, the agent had been trained on the firm’s standard document checklist. What we missed: the firm had two sub-practices (audit and tax) with different document needs. The agent defaulted to the union of both lists and sent the full superset to every new client. Two complaints, no actual damage, but a credibility hit with the partner.

• Root cause: Single knowledge base for “the firm’s onboarding process” when there were really two processes. Discovery had documented one. The team had assumed the other was obvious.
• Fix: Added a practice-area routing step at the start of the onboarding workflow. Agent now classifies the engagement type from the new-client form before sending the request. Took 2 days.
• Lesson: “The firm’s standard process” is rarely a single process in firms above 5 partners. We now ask explicitly during discovery: “How many sub-practices does this firm run, and do they have different intake?”

2. Ledger QA (Agent #4) flagged 200+ false positives on day 10. Year-end at firm #2 introduced a normal pattern (lots of large entries, lots of reclassifications) that looked like anomalies to the agent, because the eval suite had been built on a quiet month. The Slack channel where exceptions went filled up; the team stopped looking at it.

• Root cause: Eval suite was built on representative data from a calm period. We did not pull year-end data into evals because the engagement started in February.
• Fix: Added seasonality awareness, agent now factors in the period of the firm’s fiscal year before flagging. Also added a “calibrate against trailing 3 months” mode that auto-adjusts thresholds. Took 4 days.
• Lesson: Eval suites need to cover the high-variance periods of the business, not just the quiet ones. We now ask: “When is your busiest period? Show us what ‘normal’ looks like then.”

3. Monthly Reporting Analyst (Agent #5) sounded like nobody. At firm #1, the partner’s feedback after the first month-end was blunt: “This sounds like a generic AI wrote it. We have a house voice. This isn’t it.” Technically the variance analysis was correct. Stylistically, it was AI-flavored.

• Root cause: Voice training had used 6 prior monthly reports for ingestion. Not enough. The firm’s voice was more distinctive than 6 documents could capture.
• Fix: Pulled 18 months of partner-written client communications (not just reports, also emails, audit notes, advisory memos). Retrained voice. Took 1 day for ingestion, 3 days for tuning.
• Lesson: For voice-sensitive output, we now require at least 20 documents and prefer 50+. Volume matters more than recency.

What broke in week 6

Week 6 is where the operational issues surface, the ones that did not show up in eval and did not show up in early production.

1. Client Onboarding (Agent #2) kept sending follow-up nudges to clients who had already responded. The agent watched for document arrivals but its detection logic missed clients who replied via email instead of uploading to the portal. So the agent would email “we are still waiting for X” while the partner had already received and processed X.

• Root cause: Single source of truth assumed (the document portal). Reality: clients respond to wherever they got the request, which is email more often than the portal.
• Fix: Added inbox monitoring for the partner’s email, agent now matches inbound attachments to outstanding requests across both surfaces.
• Lesson: “Where does the data actually arrive?” needs to be a discovery question, not an assumption.

2. Invoice Processor (Agent #1) made the same misclassification 47 times. A new vendor pattern appeared at firm #3, a recurring monthly invoice that the agent kept assigning to the wrong GL category. The exception flow caught it, but the agent did not learn from the human correction. Each new invoice was processed identically wrong.

• Root cause: No feedback loop from human corrections back into the agent’s behavior. The 5-day discovery had not included a “this is a new pattern, learn from this correction” mechanism.
• Fix: Added a “vendor pattern learning” knowledge base that captures human corrections per vendor. Agent consults it on every invoice and applies vendor-specific rules. Took 5 days to build, integrate, and re-eval.
• Lesson: Task-tier agents need an explicit “learn from corrections” loop, not just an exception flow. This is now standard in the pack.

3. Reporting Analyst (Agent #5) over-summarized at firm #2. The variance analyses started reading like one-pagers when the partners had been used to two-pagers with more context. Technically correct, but the partners had to spend extra time adding context for client distribution.

• Root cause: We optimized for “tight, executive-summary style” but the firm’s clients expected the more discursive style.
• Fix: Added a per-firm “verbosity dial” that controls the output length and detail density. Took half a day.
• Lesson: Stylistic preferences vary by firm and even by client. The dial is now a standard configuration option.

Three redesigns we made after firm #3

After three deployments, we redesigned three parts of the pack:

1. Eval suite now explicitly includes seasonality coverage. Each agent’s eval suite must include 3+ test cases from the firm’s high-variance period (year-end, tax season, audit season, whichever applies). Adds half a day to discovery but eliminates the year-end blowup pattern.

2. Discovery now asks four questions we used to assume.

• “How many sub-practices does this firm run?” (multi-practice routing)
• “When is your busiest period, and what does ‘normal’ look like then?” (seasonality calibration)
• “Where do clients actually respond, portal, email, both?” (data-source completeness)
• “Do partners write in different voices, or is there a firm voice?” (voice training scope)

3. The retainer (Care/Grow/Scale) is now strongly recommended, not optional. Production patterns surface in weeks 4–10. Without a retainer, the firm has no scoped relationship for the fixes that week 6 needs. We discount the first three retainer months by 50% if the firm signs at engagement start (effectively, the project fee credits to retainer).

What firms keep asking for that is not in the pack

After three engagements, four requests showed up consistently. None are in the standard pack, but the data points to where Year 2 should expand:

• Audit-engagement automation. “Can the agents draft audit work papers?”, yes, technically, but audit has formal sign-off requirements we have not yet scoped. Roadmap item for Q4 2026.
• Tax-return preparation support. “Can an agent help draft tax returns?”, partial yes. Drafting the supporting workpapers is in scope; the return itself stays human-controlled for liability reasons. Add-on, ~€2,000 per firm.
• Client portal chat. “Can the agent answer clients’ simple questions about their accounts?”, yes, but this is a Tier-1 support agent (like Yiannis) bolted onto the pack. Add-on, ~€3,000.
• Practice management dashboard. “Can we see all five agents’ KPIs in one place?”, yes, this is the operating dashboard included in the pack, but we underbuilt it in v1. It is the #1 expansion request. Rebuilt for v2.

Real numbers from the first three deployments

Aggregated across the first three firms (10–15 fee earners each, mid-sized boutique):

Metric	Average	Range
Build cost (pack)	€8,500	€8,500 (fixed)
Monthly running cost	€1,050	€700–€1,400
Hours reclaimed/partner/month	28	18–42
Total hours reclaimed/firm/month	175	130–240
Time-capture leak closure	7.5%	5%–11%
Break-even month	1.2	0.9–1.8
30-day partner NPS	8.7	7–10
Tickets in first 60 days	14	9–22
Tickets that required code change	6	4–9
Tickets in days 60–90	3	1–6

The pattern: heavy support load in the first 6 weeks, then it falls off sharply. The retainer math reflects this, Care tier (€500/month) covers the post-stabilization period, where 1–3 tickets per month is the norm.

Takeaways for firm partners reading this

If you are a partner at an accounting firm considering AI agents in 2026:

1. Do not start with the pack if you cannot dedicate 5 hours/week for 4 weeks. Discovery and eval review need real partner time. Without it, the pack delivers a generic version of itself.
2. Plan for weeks 4–10 to need attention. This is when the patterns above surface. Either retain us for it or budget partner hours.
3. The biggest unlock is not the time saved, it is the time recapture for billable work. Of the 175 hours/month reclaimed, the firms that rebilled aggressively saw 5–11% revenue uplift in the first quarter. The firms that just absorbed the recaptured time saw smaller measurable impact.
4. One thing to ask any agency before signing: “What broke in your last three deployments?” If they cannot answer in specifics, they have not done enough deployments or they are not tracking honestly. Both are signals.

---

If you run an accounting firm and want to see what a customized version of this pack would look like for your practice, the discovery call is the right next step. We will tell you whether the pack fits or whether a smaller Spark engagement is the better entry point.

And if you read this and thought “I want to know what broke in your legal or e-commerce deployments too”, that is the next two field-note posts in the queue.

---

Key terms in this post: AI agent · eval suite · hallucination · AI guardrails · knowledge base · escalation rule