The ERP I built at Coen touches real money in about nine different ways: ADP-mapped payroll, Stripe payments, subcontractor invoices, customer e-signatures, change orders on an $8.5M portfolio. It also has an agentic AI tier doing takeoff analysis, contract review, scope-gap detection and recurring reporting. Those two sentences can only coexist because of one architectural rule:
Nothing an AI produces writes to the system of record without a human approving it.
Not "usually." Not "for the risky stuff." Ever. The agent drafts, flags, computes, summarizes — and then it stops at the gate and waits for a person. The person is fast because the draft is good. The company is safe because the person is there.
Why I'm rigid about this
Because I've done the other version. At Liberty Mutual in the early 2010s we automated claims workflows with RPA — the agentic AI of its day, minus the press coverage. The wins were real: errors down 15%, cycle time down 22%. But every one of those wins came after we learned, sometimes expensively, that an automated mistake is a mistake with a throughput problem. A human makes an error once. An unsupervised automation makes it four hundred times before lunch.
LLMs raise the ceiling and the floor at once. The drafts are astonishing. The failure modes are confident, fluent and occasionally fictional. That combination is precisely what a review gate is for.
How the gate actually works
- Stateless, reproducible prompts. Each agent task is a worker: same input, same output, no memory of its last mood. If a result looks wrong, I can rerun and diff it.
- The gate lives in the workflow, not in a policy PDF. The approve button is where the work happens; the unreviewed path physically doesn't exist.
- Review is designed to be cheap. Sources cited, changes highlighted, one keystroke to approve. A gate that slows people down gets bypassed; a gate that speeds review up gets defended by the very people it checks.
- Everything is logged. Who approved what, when, from which draft. When a regulator, an auditor, or an angry Tuesday asks, the answer is a query, not an archaeology dig.
The EU's high-risk AI provisions went enforceable this month, and a lot of teams are discovering that "human oversight" is now a compliance line item. Fine by me. I didn't build the gate because a law told me to. I built it because multi-hour analyst tasks compressed into minutes are only a bargain if the minutes are trustworthy — which is the whole game now.
Move fast — behind a gate. It's the only way I've found to get both.