Human-in-the-loop AI: when should an agent ask a person?

Human-in-the-loop AI is a design in which a person reviews, approves, corrects or can stop what an AI system does, instead of the system acting entirely on its own. For AI agents, the practical rule is to ask a person before actions that are irreversible, high-stakes or legally significant, and whenever the agent is stuck or unsure. EU law reinforces this: the AI Act requires effective human oversight for high-risk systems, and GDPR gives people the right to human intervention in significant automated decisions.

Key takeaways

• Human in the loop means a person can review, approve, override or stop the AI; it ranges from approving every action to monitoring with a stop button.
• Ask a person before sensitive, irreversible or high-stakes actions, and when the agent exceeds a failure or retry threshold.
• EU AI Act Article 14 requires high-risk AI systems to be designed for effective oversight, including the ability to override and stop them.
• GDPR Article 22 gives people the right not to be subject to solely automated decisions with legal or similarly significant effects, with safeguards including human intervention.
• Approval steps fail when reviewers rubber-stamp: show the evidence, limit volume and measure how often reviewers change the outcome.

What does human in the loop mean for AI?

It means a person is part of the decision path, with real authority to change the outcome. In practice there are three levels:

Level	What the person does	Typical use
Human in the loop	Approves each action before it happens	Refunds, payments, account changes, outbound emails to customers
Human on the loop	Monitors and can intervene or stop at any time	High-volume tasks with low individual risk
Human in command	Sets goals, limits and permissions, reviews outcomes	Any deployed agent

Most products mix them: an agent drafts and prepares freely, a person approves the risky step, and the team reviews samples and metrics weekly.

When should an AI agent ask a person?

An agent should ask before high-risk actions and when it is failing. OpenAI's guide to building agents names two primary triggers:

1. Exceeding failure thresholds: set limits on retries or actions; if the agent exceeds them, for example failing to understand a customer's intent after several attempts, escalate to a person.
2. High-risk actions: actions that are sensitive, irreversible, or have high stakes should trigger human oversight until confidence in the agent grows. The guide's examples are cancelling orders, authorising large refunds and making payments.

Anthropic's engineering guidance adds that agents can pause for human feedback at checkpoints or when they hit blockers, and should have stopping conditions such as a maximum number of iterations.

A practical decision list for your own product:

• Can it be undone? If not, ask first.
• Does it move money, change access or delete data? Ask first.
• Does it go to a customer or the public in your name? Ask first, at least until quality is proven.
• Does it affect someone's rights, credit, job or access to a service? A person must be able to intervene; see the legal section below.
• Is the agent unsure, looping or outside its instructions? Stop and hand over.
• Otherwise: let it act, log it and review samples.

What does EU law require?

Two pieces of EU law matter most.

The EU AI Act, Article 14, applies to high-risk AI systems. It requires them to be designed so they can be effectively overseen by natural persons while in use, with oversight proportionate to the risks, level of autonomy and context. The people assigned to oversight must be able to understand the system's capacities and limitations, stay aware of the tendency to over-rely on its output (automation bias), decide not to use it or to override or reverse its output, and interrupt it through a 'stop' button or a similar procedure. Article 26 requires deployers of high-risk systems to assign oversight to people with the necessary competence, training and authority. Whether your system is high-risk depends on its use; our high-risk AI classification checklist helps you check, and application dates have been amended since the Act was published.

GDPR, Article 22, applies whenever personal data is used. People have the right not to be subject to a decision based solely on automated processing that produces legal or similarly significant effects. Where an exception applies, the controller must still provide safeguards including at least the right to obtain human intervention, to express a view and to contest the decision. Ireland's Data Protection Commission also warns that relying on AI outputs without human review creates automated decision-making risks, and that purely automated decisions have the potential to cause harm.

How do you design a good approval step?

Design the approval so the person can make a real decision quickly. Five rules:

1. Show the evidence, not just the answer. Display what the agent intends to do, why, and the data it used, so the reviewer can check it in seconds.
2. Make the action specific. "Refund EUR 240 to order 1182 under the 30-day policy" is reviewable; "process this ticket" is not.
3. Offer approve, edit and reject, and capture the reason for edits and rejections as evaluation data.
4. Pause safely. The agent should wait without timing out into the action, and resume cleanly after approval.
5. Route by risk and confidence. Low-value, high-confidence actions can be auto-approved within limits; everything else goes to a queue.

Standards help here. MCP, the open protocol for connecting AI applications to tools, includes elicitation, which lets a server request additional information from users or ask for confirmation of an action, so approval can be built into tool calls.

What are good examples?

Some realistic patterns:

• Customer support agent: drafts replies and refunds freely; refunds above a set amount, account closures and anything mentioning legal action go to a person.
• Finance operations: extracts invoice data and matches purchase orders automatically; payments and new supplier bank details always need approval.
• Recruitment screening: summarises applications, but a person makes every shortlisting decision, because hiring decisions can have significant effects on people.
• Coding agent: proposes changes and runs tests; a developer reviews before merging. Anthropic notes that automated tests verify functionality but human review is still needed for broader system requirements (Anthropic).

What does human oversight cost?

It costs reviewer time, which you should estimate before launch. A simple hypothetical: if an agent handles 2,000 cases a month, 15% need approval and each review takes two minutes, that is 600 minutes, or 10 hours of reviewer time a month. Track the approval rate over time; as evaluation results and reviewer agreement improve, you can raise auto-approval limits step by step.

What are the pitfalls?

The main pitfall is oversight that exists on paper only. Common failures:

• Rubber-stamping. Reviewers facing hundreds of approvals approve everything. The AI Act names this tendency as automation bias. Measure how often reviewers edit or reject; a rate near zero may mean nobody is really checking.
• Approval after the fact. If the email has already gone, it is a notification, not an approval.
• Too much agency. OWASP lists excessive agency as a top LLM risk: tools with broader permissions than needed make every missed approval more damaging.
• No stop button. Make sure someone can pause the agent for everyone, quickly, and that the team knows how.
• Untrained reviewers. Reviewers need to know the system's limits and the policy they are applying.

Build the approval points into your LLM evaluation too: include test cases where the correct behaviour is to stop and ask.

Frequently asked questions

Is human in the loop a legal requirement?

For high-risk AI systems under the EU AI Act, effective human oversight is required by Article 14. Under GDPR Article 22, people have rights around solely automated decisions with significant effects, including human intervention where exceptions apply. For other uses it is good practice rather than a specific legal duty.

What is the difference between human in the loop and human on the loop?

In the loop, a person approves each action before it happens. On the loop, the system acts by itself while a person monitors and can intervene or stop it. Many products use both, depending on the risk of each action.

Does human review make an AI system slower?

For the actions that need approval, yes, which is why approval should be limited to risky or uncertain cases. Everything else can run automatically with logging and sampling, so most users never wait.

What is automation bias?

Automation bias is the tendency to accept a system's output without enough scrutiny. The EU AI Act names it in Article 14 and requires that people overseeing high-risk systems are made aware of it. Showing evidence and tracking override rates helps counter it.

How do I know when to remove an approval step?

When evaluation results and real approval data show the agent is consistently right for that action type, and the cost of a rare error is acceptable and reversible. Raise auto-approval limits gradually and keep sampling outcomes.

If you are designing approval steps for an AI agent, book a free 30-minute call to talk through the options.

Cover photo by RDNE Stock project on Pexels.