Most AI projects fail at deployment, not at demo. theagency47's 5-phase methodology (Discover, Design, Build, Deploy, Optimize) exists to push the failure-prone moments forward, into the cheap-to-fix phases, so the expensive phases (build, deploy) carry only solved problems.
theagency47 uses a 5-phase methodology (Discover, Design, Build, Deploy, Optimize) that runs 14 to 90 days depending on engagement tier. Each phase has explicit gate criteria. Discovery produces an agent shortlist with ROI estimates. Design produces a one-page signed spec per agent. Build delivers an agent passing 90%+ on an eval suite of 20+ test cases. Deploy includes human-in-the-loop monitoring for the first 14 days. Optimize tunes the KPI dashboard weekly until performance is stable, then hands the agent over to the client. The specs are the unit of work, the eval suite is the unit of quality, and the KPIs are the unit of value.
theagency47 · Updated May 2026Goal: Pick the right agents to build.
We walk through your processes, what you do, where the unbillable hours live, where errors cluster. Output is a shortlist of candidate agents, each scored on three dimensions: hours reclaimed, error rate reduction, throughput gain. We do not build agents that fail any of the three. You can simulate payback period for the "hours reclaimed" dimension before the call.
Gate: Client sign-off on the shortlist.
Goal: Make the work specifiable.
A one-page spec sheet per agent: job-to-be-done, inputs, outputs, tools, autonomous decision boundaries, escalation rules, KPIs, eval cases. This is the single most under-rated phase in agent work. Specs prevent scope creep, anchor refunds, and force "vibes" into "verifiables."
Gate: Signed spec sheet per agent.
Goal: Make the agent pass evals.
Two-week sprint per agent. Daily progress updates. Eval suite of 20–30 test cases built alongside the agent itself, not after. Common-case, edge-case, and adversarial inputs. Agents must achieve 90%+ pass rate before they leave this phase.
Gate: Eval suite passes ≥90%; client reviews output samples.
Goal: Run safely in production.
Production cutover with human-in-the-loop monitoring for 14 days. Every agent action is reviewable by a human before downstream effects. Team training session delivered. Monitoring and alerting wired before traffic flips.
Gate: 14-day in-production pass rate ≥95% with human oversight.
Goal: Hand over a stable system.
KPI dashboard tuned. Weekly optimization until performance is stable across two consecutive reporting cycles. Agent moves from human-in-loop to autonomous-with-sampling. Documentation finalized. Handover ceremony.
Gate: Two consecutive stable reporting cycles. Client signs handover.
Once handover closes, clients can opt into a monthly retainer for ongoing optimization, health checks, and new agent builds. About 70% of projects convert to a retainer within the first 30 days post-handover.
The default AI agency pattern is "let's try a prompt and see what happens." This produces demos but not deployments. We refuse to write a prompt until a one-page spec is signed. This shifts the riskiest decision (what is this agent actually for?) into the cheapest-to-revise phase.
"We'll improve it in production" is how AI projects fail. We build the eval suite alongside the agent and gate launch on pass rate. The eval suite then re-runs automatically after every change in production. If an update breaks an eval, the update reverts. This is what "production-grade" actually means.
The default agency pattern is "we'll deploy and then bill you forever for maintenance." We deploy, optimize until stable across two consecutive cycles, then hand over to your team with the source files. About 70% of clients then opt into a retainer, but it is a choice, not a captivity. Ownership is yours from day one.
theagency47 standardizes on a small, opinionated stack so we ship faster and clients can take ownership without a steep learning curve. For the broader frame of how this methodology maps to the three levels of AI in a business, and to why we built theagency47 as a delivery agency rather than a consultancy, see those pages.
The full stack is documented in the engagement contract. Clients receive the full source (prompts, configs, KB schemas, eval cases, integration code) at handover.
Methodology means what we exclude as much as what we include. Five things theagency47 does not do:
Spark = 14 days · Workforce Starter = 30 days · Workforce Pro = 60 days · Enterprise = 90+ days. Each phase is gated. See pricing for tier details.
One page, eight fields: job-to-be-done, inputs, outputs, tools available, autonomous decision boundaries, escalation rules, KPIs, eval cases. Client signs before any building.
20–30 test cases per agent with known expected outputs. Split common-case (70%), edge-case (20%), adversarial (10%). Pass rate ≥90% required to deploy. Re-runs automatically after any production change.
For 14 days after deployment, every agent action is reviewed by a human before downstream effects. After 14 days, if pass rate exceeds 95%, agent moves to autonomous mode with sampling-based audit.
Automatic revert to the last passing version. The change that broke the eval is investigated. Production fixes go through the same eval gate as initial deployment.
Same five phases, smaller scope. New agents added under a retainer follow the full Discover → Optimize cycle, just compressed (typically 5–10 days for a task-tier agent under retainer).