How to train an AI agent for your business.

1. What "training" actually means in 2026

The word "training" is the most overloaded term in AI right now. In academic and research contexts, it means running gradient descent over billions of parameters on a GPU cluster. In business contexts, it almost never means that. When a vendor tells you they will "train your AI agent," they mean four specific things:

• System-prompt customization. The agent's instructions (role, voice, scope, escalation rules) are written specifically for your business.
• Knowledge-base curation. The agent is given access to your documents (policies, FAQs, product specs) and retrieves from them at decision time.
• Tool integration. The agent is wired to the APIs and systems it needs (Gmail, Shopify, CRM, internal DB) so it can actually do work.
• Eval-case construction. The agent is tested against 20–40 representative inputs, with expected outputs, before it goes live and after every meaningful change.

That is it. No fine-tuning, no retraining the base model, no millions of training examples. The base model (we standardize on Claude) already knows how to read, reason, and write. The "training" is teaching it the specifics of your business so it produces correct work on your specific inputs.

True model fine-tuning is still rare in 2026. The combination of strong base models plus prompt-level customization plus retrieval covers ~95% of business workflows at a small fraction of the cost. Reserve fine-tuning for edge cases where a base model genuinely cannot acquire a skill from instructions, and even then, validate with the four components above first.

3. The 7-phase training methodology

The sequence we run for every agent build. Each phase has a deliverable, an owner, and a duration. We do not move to the next phase until the deliverable is signed off.

Phase 1, Workflow discovery (1–2 days)

Capture the workflow on paper before anyone touches a model. The deliverable is a 1–2 page document covering: what triggers the workflow, what the decision rules are, what the most common exceptions look like, what "correct output" actually means, and what volume looks like (per day, per week).

Most teams skip this. Most teams regret it. Discovery is the cheapest phase by hours and the most expensive phase by leverage, every hour spent here saves three in phase 5.

Phase 2, System prompt design (2–3 days)

Translate the discovery doc into a system prompt. A well-formed system prompt for a production agent is typically 800–2,000 words, structured into: role and goal · voice and tone · scope (what the agent does) · anti-scope (what it never does) · decision rules · escalation triggers · output format.

Iterate the prompt against a handful of sample inputs until the agent produces the right shape of output ~80% of the time. Do not chase 95% in this phase, that comes from the eval suite in phase 5.

Phase 3, Knowledge base curation (1–3 days)

Decide what the agent needs to read at decision time. For small scopes (under ~10 pages of reference material), put it directly in the system prompt, simpler, faster, no retrieval failure modes. For larger scopes (FAQs with hundreds of entries, product catalogs, policy libraries), set up retrieval-augmented generation (RAG) or use a tool function that fetches the right doc on demand.

The most common knowledge-base mistake is throwing 200 PDFs at a vector store and assuming retrieval will sort it out. It will not. Curate. When knowledge is your real bottleneck across several agents, build it as a shared foundation first, a Second Brain with Obsidian that every agent reads from and writes back to.

Phase 4, Tool integration (2–5 days)

Wire up the APIs the agent needs to call to actually do work: send email, look up an order in Shopify, write to a CRM, post to Slack, generate a shipping label, issue a refund. Each tool is a small, well-typed function with a clear name, parameters, and return value. Six to ten tools is typical for an operational agent; two to four for a task agent.

Two principles: every tool has read-only siblings (the agent can look up an order before deciding to refund it), and high-impact tools (refunds, sending mail to customers) gate behind a confirmation step in early production. For more on tooling and frameworks, see Custom AI Agent vs ChatGPT Team vs Claude Projects.

Phase 5, Eval suite (2 days)

Build 20–40 test cases. Each case is an input the agent will face in production plus the expected output (or the expected kind of output, when exact match does not make sense). The suite covers three buckets: routine inputs (the agent should breeze through these), edge cases (where the rules are ambiguous), and red-flag inputs (where the agent should escalate, not act).

Run the suite after every system-prompt change. Track the pass rate as a single number. We deploy when the eval pass rate is ≥90% across the suite, with no failures in the red-flag bucket. The full methodology is in our eval guide.

Phase 6, Human-in-the-loop production (5–10 days)

Run the agent against real inputs with progressively less human oversight:

1. Shadow mode (2–3 days). Agent drafts outputs, a human reviews every one, the agent does not act. Goal: catch failure modes that the eval suite missed.
2. 100% human approval (3–5 days). Agent acts but a human approves before each send. Goal: build team trust and catch tone/judgment issues.
3. Sampled review (ongoing). Agent acts autonomously on routine inputs. Human reviews 10–20% of outputs and 100% of escalations. Goal: keep quality stable while reducing oversight cost.

Skipping this phase to "go live" faster is the single most common deployment mistake we see. A one-week patient handover beats a one-day enthusiastic launch followed by a six-week post-mortem.

Phase 7, Continuous improvement (ongoing)

Once a week, review the logs. Three buckets to inspect: escalations (what got bumped to humans, and why), anomalies (inputs the agent had not seen before), and disagreements (where the human overrode the agent's decision). Each is a signal to update either the system prompt, the knowledge base, the eval suite, or all three.

A well-tuned operational agent climbs from a ~65% auto-resolution rate in week 1 to 80%+ by month 2 through this loop. Skip the loop and the agent silently regresses against drifting business conditions.

5. What to do yourself vs. what to hire for

An honest split for an operator deciding whether to in-source or bring in help:

Phase	Hire?	Reasoning
1, Discovery	Do yourself	Nobody knows your workflow better than your team.
2, System prompt	Hire if first agent; in-house after 3rd	The prompt-engineering pattern library matters early. After three builds, your team has it.
3, Knowledge base	Do yourself	Curation is editorial work, not technical. Your team is better at it than any outsider.
4, Tool integration	Hire (or have a developer)	This is software. Skip the pretence of no-code on production-grade work.
5, Eval suite	Mixed: cases yourself, framework hired	You know what "correct" looks like. The eval harness is a one-time build.
6, Human-in-the-loop	Do yourself	The team that will use the agent has to approve it. Outsourcing this defeats the point.
7, Continuous improvement	Hire (retainer) or in-house once mature	The first 90 days produce the biggest learning gains. Retainer pays back at this stage; later it depends on volume.

If you want a faster scoping conversation about which phases you should run in-house vs. hand to us, the €297 AI Audit covers exactly that question in 30 minutes.

7. The minimum viable training plan

If you wanted to ship one agent in the next 30 days using this methodology (without overthinking it) this is the irreducible plan:

1. Week 1. Pick one workflow (the noisiest one in your team's day). Write the discovery doc, 1–2 pages, in plain language.
2. Week 1–2. Draft a system prompt against the doc. Iterate against 5 sample inputs until output shape looks right.
3. Week 2. Curate the knowledge base. Resist the urge to upload everything.
4. Week 2–3. Wire up the 4–6 tools the agent needs. Use a developer for this step or accept that it will take longer.
5. Week 3. Write 20 eval cases. Run them. Iterate until ≥90% pass.
6. Week 4. Shadow mode for 2 days. Human-approval mode for 3 days. Sampled review thereafter.
7. Ongoing. Two hours a week reviewing logs. Update prompt / KB / evals based on what you see.

That is the entire methodology in seven bullets. It works for fashion brands, accounting firms, B2B SaaS support teams, and law firm intake processes, we have shipped it across all four in 2025–2026. The technology is not the hard part. The discipline of running the seven phases instead of cutting corners is.