AI governance
How to Pilot AI Agents Safely in a Regulated Business
Last updated
The short answer
Pilot an AI agent on one well-understood workflow, with real but limited data, under rules you have written down before it starts. Run it in shadow mode first, where it proposes and a person decides, then let it act on low-risk items while a person approves anything above a set threshold. Log every decision, measure against a baseline taken before the pilot, and agree in advance what result means scale, adjust or stop. In a regulated business, the approval rules and audit trail are part of the pilot, not something added later.
Step 1: Scope narrowly
- Pick one workflow with a clear owner, steady volume and outcomes you can check, for example claims intake triage, invoice coding or privacy request handling.
- Define what the agent may do, what it may only propose, and what it must never touch.
- Set a time box, typically a few weeks of live operation, and the success measure.
Step 2: Settle data and risk before building
- Confirm the lawful basis for processing the data and record it in your data protection impact assessment if one is needed.
- Give the agent its own identity and the least access it needs. Never let it run on a person's credentials.
- Classify the workflow under your AI policy and, if it serves EU users, check whether it could be high-risk under the EU AI Act. Uses in employment, credit and some insurance decisions are the usual candidates.
- Write the approval rules: which actions need a person, above which value or risk, and who that person is.
Step 3: Run in stages
| Stage | What the agent does | Move on when |
|---|---|---|
| Shadow | Proposes actions on live items; a person does the work as usual and compares | Its proposals match reviewers' decisions on a large enough sample |
| Assisted | Acts on low-risk items; a person approves everything above the threshold | Error and override rates stay within the agreed limits |
| Supervised | Handles the normal path; a person reviews exceptions and samples the rest | The pilot measure is met and the owner signs off |
Step 4: Keep the evidence
- Log every input, action, policy check and approval, so any single decision can be reconstructed for an auditor or regulator.
- Track overrides: every time a person changes the agent's proposal is a signal about where it is weak.
- Keep the baseline and the pilot results side by side for the decision meeting.
Step 5: Decide
At the end of the time box, compare the result with the measure you agreed. Scale if it was met with acceptable risk, adjust and rerun if it was close, and stop if it was not. A stopped pilot with good evidence is a success: it saved you from scaling the wrong thing.
Frequently Asked Questions
- What is shadow mode for an AI agent?
- The agent processes real items and records what it would do, but takes no action. People continue to do the work, and their decisions are compared with the agent's. It is the safest way to measure accuracy on real data.
- Who should approve an AI agent's actions?
- The person who would normally be accountable for that decision, such as a claims handler, a finance approver or a privacy officer. Approval thresholds should mirror the authority limits people already work under.
- How long should an AI agent pilot run?
- Long enough to see a representative mix of cases, including month-end peaks or seasonal spikes if the workflow has them. For most workflows that means several weeks of live operation after any shadow period.
Keep exploring
Related reading
Next step
Get a measured answer for your workflow
We map one process, measure what it costs today, and give you a fixed proposal.