Deploying AI Agents in Production With Human Approval: HITL Patterns That Survive

Deploying AI agents in production with human approval means running agents whose externally visible actions pause for a person before they take effect. Human-in-the-loop, in this sense, is not a person watching a log — it is a person required at specific, named steps of the agent's work. In production the pattern has to survive four things a notebook demo does not: a pause that outlives the process, many agents with many pending items at once, a rule for which items may skip the human, and a record of who decided what against which version of the agent. Vocion, an open-source agent workforce platform, implements these as approve and ask workflow steps, a requiresApproval flag on plugin-provided skills, trust.yaml confidence thresholds, an autonomy ladder on missions (levels 1 to 5), and a unified review queue with a REST API. Everything is authored as YAML and Markdown in git.

This guide is for an engineer who has a working prototype agent and has been told it cannot touch a customer, a CRM record, or an outbound email without a person signing off. By the end you can name the five places a human can be inserted, pick the right one per action, wire an approval gate into a workflow, and drive an approval decision over HTTP.

What breaks when you skip the human

Three failure modes show up the first week an agent runs unattended in production. First, an irreversible external action — an email sent, a CRM record updated, a kit rejected — happens on bad judgement with nobody positioned to catch it before the fact. Second, nobody can say which prompt, skill, or model produced a given output six months later, so a bad decision cannot be traced back to what caused it. Third, the run has nowhere to wait: a Python interrupt() in a notebook has one graph, one thread, and dies with the process, which is fine for a demo and wrong for a system that has to survive a redeploy.

Production human-in-the-loop has to solve all three at once: a pause that survives a restart, a queue a team can actually work through, and an audit row tying every action back to the workspace that authored it.

It also helps to separate human-in-the-loop from human-on-the-loop, since the two get used interchangeably. Human-in-the-loop, as this guide uses it, means the run cannot proceed past a named step without a person's decision — the approve step below is a hard stop. Human-on-the-loop is weaker: a person can intervene, but the system proceeds without one by default. Vocion's trust.yaml rules and mission autonomyPolicy levels 3 and up are closer to human-on-the-loop — the action executes unless a rule or a task flags it — while an approve step and a requiresApproval skill are human-in-the-loop in the strict sense. Both are legitimate; the mistake is not knowing which one a given action is actually getting.

Where you can put a human: five insertion points, ranked by cost

Vocion gives you five places to require a person, from cheapest (blocks one step) to most expensive (blocks a whole mission's autonomy):

1. An approve step inside a workflow. Pauses one run in the review queue for a single yes/no decision on a named earlier step's output.
2. An ask step inside a workflow. Pauses until a human supplies text — unless a default already resolves to something, in which case the step completes without pausing at all.
3. requiresApproval on a plugin-provided skill. A skill's operation is marked so its output always lands in the review queue rather than executing directly, regardless of which workflow or mission called it.
4. A trust.yaml confidence threshold. A per-action rule that lets a proposal through without a person only above a stated confidence, and only when explicitly enabled.
5. A mission's autonomyPolicy.level. A blanket policy on how much an entire standing responsibility may do without asking, from "draft only" to "manage a goal."

Each is documented as its own authored entity in vocion-core, at vocion-core/docs/entities/workflow.md, docs/entities/trust.md, and docs/entities/mission.md. The plugin contract for requiresApproval lives at packages/core/src/libs/plugins/loader.ts and packages/core/src/libs/plugins/registry.ts.

How do I add an approval gate to a workflow?

A workflow is "a deterministic procedure: the same structure on every run," per docs/entities/workflow.md. It has four step types — sync, ask, approve, action — and that is the whole vocabulary; there is deliberately no "call a skill" step, because skills are read by the agent on its own judgement rather than sequenced by the runtime. The approve step takes a prompt and an optional reviews field naming the earlier step whose output is being judged:

# workflows/discovery-followup/workflow.yaml
slug: discovery-followup
name: Discovery Follow-up
description: Turn a discovery call into an approved follow-up email.
agent: revenue-lead
trigger:
  type: schedule
  cron: '0 12 * * 1-5'
steps:
  - name: refresh-mail
    type: sync
    sources: [gmail]
  - name: transcript
    type: ask
    prompt: Paste the discovery call transcript.
    default: '{{input.transcript}}'
  - name: review-draft
    type: approve
    prompt: Approve the follow-up email before it sends.
    reviews: transcript
  - name: send
    type: action
    action: gmail.send
    input:
      body: '{{steps.review-draft.output.body}}'

This is the example workflow from docs/entities/workflow.md, quoted verbatim. Notice the ask step's default: '{{input.transcript}}' — when that interpolation resolves to a non-empty string (because an automation supplied it), the step completes immediately and never pauses. The same workflow file serves an automated caller and a person starting it by hand. The send step, an action type, runs a registered connector action (gmail.send); the source of truth for registered action ids is packages/core/src/libs/actions/.

There is no built-in way for a workflow step to skip a person by itself. Approval is enforced at the step level; a skill or playbook referenced inside the run cannot grant itself sending rights (docs/entities/trust.md).

How does the run survive the pause?

A workflow's runtime is WorkflowService.runLoop, applied to a workflow table (docs/entities/workflow.md). The stack underneath it is "a Next.js app + Postgres schema + MCP server + workflow runner" (vocion-core/README.md), and the local dev stack brings up "Postgres + Langfuse + Temporal" (README.md line 103, npm run dev:up in docs/getting-started.md). Automations — the entities that fire a workflow on a schedule or an event — resolve to "a Temporal schedule or event match" (docs/object-model.md). A paused run is not a suspended process; it is a row a durable scheduler can resume on a signal, days later, after a redeploy, regardless of which process paused it.

This is the fair contrast with a single in-process interrupt(): LangGraph's interrupt mechanism (documented at https://docs.langchain.com/oss/python/langgraph/interrupts) is a real durable pause when it is backed by a persistent checkpointer — it is not a toy. The operational difference is scope: one graph and one thread versus many agents, many workflows, and one review queue that every pending item from every one of them lands in. Temporal itself makes the same durability case independent of any particular agent framework (https://temporal.io/solutions/ai); Vocion's runner sits on it rather than reimplementing scheduling and retries.

How does a human actually approve — dashboard or API?

Every pending item — a paused workflow run, a mission awaiting review, or a pending action proposal — lands in one place: /dashboard/review. The same surface exists as a REST API, verified in packages/core/src/app/api/v1/reviews/.

Read the queue:

curl -H "Authorization: Bearer $VOCION_TOKEN" \
  "https://your-install/api/v1/reviews?kind=workflow&includeSnoozed=false&limit=25"

GET /api/v1/reviews (packages/core/src/app/api/v1/reviews/route.ts) accepts kind (workflow | mission | action, to see one plane only), assignedTo (a user id, or unassigned for triage), includeSnoozed, limit, and offset, and returns a real total. Auth is a tenant API token (Authorization: Bearer vcn_live_…) or a signed-in dashboard session — both resolve through the same authApi() call.

Decide an item:

curl -X POST -H "Authorization: Bearer $VOCION_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"kind":"workflow","id":482,"action":"approve"}' \
  "https://your-install/api/v1/reviews/decide"

POST /api/v1/reviews/decide (packages/core/src/app/api/v1/reviews/decide/route.ts) takes { kind, id, action, reason?, editedInput? }, where action is approve or reject. Deciding requires the approve capability. editedInput is the edit-then-approve payload: corrected fields for a pending action, or the input a paused workflow resumes with; it is ignored on reject. This is the exact call a human's review UI makes when someone clicks "approve" on that follow-up email — there is no separate internal API the dashboard has that an external client cannot reach.

Tokens are minted from the dashboard at /dashboard/api-tokens, not from shell access — see /blog/the-review-queue-over-http for the full endpoint list this API grew in v2.24.0, including GET /api/v1/reviews/auto-executed, POST /api/v1/reviews/propose, and the reviewer-feedback and learning-candidate surface described below.

Which actions should skip the queue?

Not every action needs a person every time. trust.yaml, one file per workspace, is "the one place that says which proposed actions may execute without a human looking first, and how confident the system has to be" (docs/entities/trust.md):

# trust.yaml
rules:
  - action: hubspot.update
    autoApproveAbove: 0.95
    enabled: true
  - action: gmail.send
    autoApproveAbove: 0.99
    enabled: false

Each rule names a registered action id (hubspot.update, discovery.review_proposal, personalization.enroll, qc.hold, qc.release, qc.request_rework, dataset.add_example, and others in packages/core/src/libs/actions/), an autoApproveAbove threshold from 0 to 1, and enabled, which defaults to false. A rule with enabled: false never auto-approves whatever the threshold says — flipping it off reverts the rule without deleting it. A proposal that clears the threshold still executes through ActionService's auto-approval check, and it still lands in the review queue's auto-executed list, retrievable at GET /api/v1/reviews/auto-executed — nothing that skips a human skips the audit trail. The docs are blunt about the intended discipline here: "Keep the list short and the thresholds high."

How much autonomy should a mission have?

A mission is different from a workflow: it is "a standing responsibility owned by one agent: the goal, what good looks like, and how much freedom the agent has" (docs/entities/mission.md), with no fixed procedure. autonomyPolicy.level is an integer 1 to 5, quoted verbatim from the schema docs:

1 draft only, 2 ask before action, 3 act within rules, 4 manage a goal, 5 improve itself. Levels 1–2 gate every external action; 3+ let them run unless the task flags approval. Internal analysis and drafting are never gated.

A level-2 mission — the example in docs/entities/mission.md is a pipeline-health mission at level 2 — can read data, form a judgement, and write a brief on its own, but any action that leaves the workspace (a CRM update, an email) still queues for a person. This is a blanket dial for a whole standing responsibility, distinct from trust.yaml, which is scoped per action; a level-3 mission that manages its own goal can still have every gmail.send proposal it makes held by a trust.yaml rule with enabled: false. Treat the two as independent controls, not substitutes for each other.

Who is accountable when a team of agents shares the queue?

A team groups agents "under a lead" and names "the human accountable for its work" (docs/entities/team.md). The accountableUser field is an authored email address, resolved to a user id at apply time, not a convention someone has to remember: a team can inherit it from workspace.yaml or set its own. When a mission's lead hands work to a specialist, or a workflow names its owning agent, the accountable human for whatever lands in the review queue is a field in the workspace, not tribal knowledge.

How does a human's decision make the agents better?

An edited approval is not a one-off correction. docs/entities/learning-step.md describes a learning step as "a named bucket of rules an agent reads" — rendered to /learnings/<name>.md in the agent's virtual filesystem, and whitelisted per-agent in the workspace so it doesn't turn into a junk drawer. The feedback loop that fills those buckets runs through a learning_candidate row: per /blog/the-review-queue-over-http, "the feedback worker classifies a job and, when the classification proposes a rule, records a pending candidate. It still never writes a live rule: a person approves it into one or rejects it with a reason, and the reason is kept." A reviewer overriding a draft does not silently retrain anything; it produces a candidate rule that waits for the same kind of decision an approve step asks for.

How do I prove later which agent version approved what?

Every workspace:apply records a row in workspace_version: git SHA, applied_at, files touched, per-resource counts, and who applied it (docs/workspace.md, "Audit trail"). Every tool_call stamps a workspace_sha, so "why did the agent draft the email like that" is answerable months later:

SELECT tc.agent_slug, tc.tool, tc.input, tc.output, tc.workspace_sha, tc.created_at
FROM tool_call tc
WHERE tc.id = <row_id>;

-- then `git show <workspace_sha>` in the workspace's repo to see the exact
-- prompts + skills active at the moment the call ran.

The workspace_sha shape itself tells you how clean the apply was: a bare git SHA is a clean apply; <sha>-dirty-<hash> means uncommitted changes were live at apply time; local-<hash> means the workspace was not in a git repo at all. Whatever an agent did, it traces back to the exact files that were in effect when it did it — including which approve gates existed and which trust.yaml thresholds were active.

What this costs you

Queue latency is real: an approve step blocks whatever comes after it until someone looks, and a schedule-triggered workflow that fires overnight but waits on a person who only checks the queue once a day inherits that person's schedule as an unwritten SLA. Reviewer fatigue is real too — a long queue checked in a hurry gets rubber-stamped. The honest failure modes worth watching for: threshold creep (raising autoApproveAbove because the queue is annoying, not because the action got safer), a reviewer who is on holiday with nobody covering assignedTo, and an ask step whose default silently resolves to something nobody meant to supply, skipping a pause that was supposed to be a checkpoint. trust.yaml thresholds are per action, not per agent — an agent that has earned trust on hubspot.update has earned nothing on gmail.send, and the schema does not let you shortcut that.

None of this is free to run, either. Someone has to own the queue: a team with an accountableUser and nobody actually checking /dashboard/review on a cadence is a queue that fills up quietly while workflows sit paused. Budget for that ownership before the first approve step ships, not after the first missed deadline traces back to an unread review item.

Run it yourself

@vocion/core is not published to npm; clone the repo (https://github.com/vocion/vocion-core) and follow docs/getting-started.md. Step 11 builds the discovery-followup workflow above with its approve gate; step 12 adds the trust.yaml from this guide. Bring the stack up with npm run dev:up (Postgres, Langfuse, Temporal), validate with npm run workspace:check -- <path>, then write it with npm run workspace:apply -- <path> --project <id|slug>. Trigger the workflow by hand — POST /api/v1/workflows/discovery-followup or /dashboard/workflows — and read the paused item back with the curl calls above.

Related

Read the full step-by-step build in Workflows and the endpoint list this review API shipped with in The review queue over HTTP.

Start with a workflow that has one approve step and a trust.yaml with nothing enabled — earn the auto-approvals later, one action at a time.