Human-in-the-Loop
A deployment pattern where every AI agent action is reviewable by a human before downstream effects, used during the first weeks of production.
Also known as: HITL · human-on-the-loop · supervised deployment
What is Human-in-the-Loop?
Human-in-the-loop (HITL) is a deployment pattern in which a human can review every AI agent action before it has downstream effects. After each launch, theagency47 runs a HITL window: the agent drafts, a person approves, and the pass rate builds up under real traffic. Once it meets the agreed threshold, the agent moves to autonomous mode with sampled audits.
Christos Papadimitriou, theagency47 · Updated October 20, 2026How HITL works in practice
For the first 14 days after deployment:
- The agent processes inputs and produces a draft output.
- The draft is queued for a human reviewer (assigned per agent).
- The human approves, edits, or rejects.
- Approved drafts go out; edited drafts go out with the edit; rejected drafts feed back into eval refinement.
After 14 days:
- With a high pass rate on real tasks during the 14-day review, the agent moves to autonomous mode for that category of action.
- If the pass rate falls short, HITL continues for another period and we investigate failures.
- High-stakes actions (refunds above policy, external communications to executive contacts) stay HITL forever, regardless of pass rate.
Clients approve agent replies through approvals in OS47.
Why HITL is non-negotiable
Without HITL, the first production week of any AI agent surfaces issues that did not show up in eval, new input patterns, integration quirks, edge cases of business logic. HITL turns those into approval-queue items rather than production incidents.
FAQ
Does HITL slow down the agent?
For the first 14 days yes, by adding human review time. But humans review faster than they produce, so the team is still net-faster than baseline.
What if there is no human available to review?
Then the agent waits, or escalates differently. We design queues with timeout escalation. No silent failures.
Related terms
Bounded Autonomy
The engineering pattern of giving an AI agent specific permissioned actions and explicit escalation rules, letting it act independently within tight, defined boundaries.
Escalation Rule
A configured condition that hands an agent task off to a human when the agent is uncertain, out of scope, or facing high-stakes input.
Eval Suite
A structured set of test cases with known expected outputs that verifies an AI agent's behavior before deployment and after every change.
