# Human in the Loop: Designing Approval Gates for AI Agents

URL: https://andreasindorf.com/en/human-in-the-loop-ai-agents/
Author: Andreas Indorf · Published: 2026-11-07 · Updated: 2026-11-07 · German original: https://andreasindorf.com/human-in-the-loop-ki-agenten/

> When AI agents need human approval, how to design thresholds, sampling, escalation and a kill switch, and what Article 14 of the EU AI Act requires.

AI agents write quotes, answer customer emails, book invoices and change code. The more they do on their own, the more important a question becomes that many projects only ask after the first mistake: at which point does a human decide, and how do you make sure that decision is more than a click? This article shows when human approval is necessary, how to design approval gates and what the EU AI Act says about it.

**In short:** Human approval belongs in front of every action that is hard to reverse, has an external effect, moves money or directly affects people. How strictly things are checked is set per task through autonomy levels: from suggestions, through approval of every action and sampling, to monitoring via metrics. Good approval gates have clear thresholds, escalation paths, a kill switch and a complete audit log. The biggest danger is automation bias, meaning habitual rubber-stamping, and it can be measured.

## What "human in the loop" really means

The term is often used as if any human anywhere in the process were enough. In fact, it covers three different forms:

- **Human in the loop:** The agent stops, a human decides, and only then does the process continue.
- **Human on the loop:** The agent acts on its own, a human monitors and can intervene.
- **Human out of the loop:** The agent acts without ongoing human control.

For companies, the question is not which form is right, but which form fits which task. In its article [Building effective agents](https://www.anthropic.com/engineering/building-effective-agents), Anthropic describes how agents can pause for human feedback at checkpoints or when they hit blockers, that stop conditions such as a maximum number of iterations are common, and that greater autonomy can bring higher costs and compounding errors. An approval gate is exactly this checkpoint, set deliberately rather than by chance.

## When human approval is needed

Not every action needs approval. Too many gates create review fatigue; too few create loss of control. Four criteria help you decide:

1. **Reversibility:** Can the action be undone with reasonable effort? An internal note, yes; a customer email that has been sent, no.
2. **External effect:** Does the action reach customers, suppliers, authorities or the public?
3. **Financial effect:** Does it trigger payments, orders, credit notes or price commitments?
4. **Impact on people:** Does the decision directly affect applicants, employees or customers?

If one of these criteria applies to a high degree, an approval gate belongs in front of the action. Under [Excessive Agency](https://genai.owasp.org/llmrisk/llm062025-excessive-agency/), the OWASP project explicitly names human approval of high-impact actions, such as sending or publishing content or deleting data, as a countermeasure. Just as important: authorisation must be checked in the target system, not by the language model itself.

## Autonomy levels: dosing approval per task

A four-level model has proven useful; you set the level per task, not per agent. The same agent can produce an internal summary on its own and still have to obtain approval for every customer email.

| Level | Agent | Human | Typical approval gate |
|---|---|---|---|
| 1 – Suggest | creates drafts and analyses | decides and executes | no action without a human |
| 2 – Prepare with approval | fully prepares the action | reviews and approves every action | gate before every execution |
| 3 – Execute with sampling | acts within fixed limits | checks samples and exceptions | gate when a threshold is exceeded |
| 4 – Autonomous with monitoring | acts on its own, reports anomalies | monitors metrics, steps in on alerts | alert and kill switch |

New tasks start at level 1 or 2. A task is promoted only when quality has been stable over a defined period; it is demoted immediately if quality drops. How the levels fit into the overall framework for agents is described in the guide [AI agents in business](https://andreasindorf.com/en/ai-agents-in-business/).

## Designing approval gates: the five building blocks

### 1. Thresholds

Thresholds turn a blanket approval into a targeted one. Typical thresholds are amounts (credit notes above a fixed value), recipients (new customers rather than existing ones), content (prices, deadlines, legal statements) and uncertainty (the agent found no suitable source). Below the threshold, the agent may act at level 3; above it, the gate applies.

### 2. Sampling

At level 3, a human does not check every action but a sample. Set the proportion, draw the sample at random and add targeted checks of conspicuous cases. A sample that the agent effectively picks itself is not a sample.

### 3. Escalation

Every gate needs a clear answer to the question of what happens if nobody decides. Define deputies, deadlines and a safe default state: nothing goes out without approval. Difficult cases need a second route, for example to the head of the department or to the legal team.

### 4. Kill switch

Every agent and the system as a whole must be able to be stopped immediately without half-finished operations being left in an undefined state. Technically, this means actions are designed so that they are executed either completely or not at all, and the kill switch has been tested, not merely installed.

### 5. Logging

Every action, approval, rejection and correction is recorded with timestamp, person and reason. The log is your evidence towards customers, auditors, data protection officers and the Betriebsrat (works council), and it is the data basis for the metrics further below. How co-determination comes into play here is explained in the article [AI and the works council (in German)](https://andreasindorf.com/ki-betriebsrat/).

## What the EU AI Act requires

The [EU AI Act](https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng) explicitly regulates human oversight for high-risk AI systems, for example in HR decisions or creditworthiness assessments. Two articles are central:

**Article 14 (human oversight)** requires high-risk systems to be designed so that natural persons can effectively oversee them while they are in use. The people overseeing them should, among other things, understand the capabilities and limitations of the system, remain aware of the possible tendency to rely automatically or excessively on its outputs, interpret the outputs correctly, be able to decide in any particular situation not to use the system or to override its output, and be able to interrupt the system through a "stop" button or a similar procedure.

**Article 26 (obligations of deployers)** is aimed at companies that use such systems. They must assign human oversight to people who have the necessary competence, training and authority, monitor operation on the basis of the instructions for use, and keep the automatically generated logs, to the extent these are under their control, for at least six months. Article 12 additionally requires high-risk systems to be technically capable of automatically recording events over their lifetime.

On timing: under the [Digital Omnibus Regulation (EU) 2026/1744](https://eur-lex.europa.eu/eli/reg/2026/1744/oj/eng), the high-risk obligations apply from 2 December 2027 for systems under Annex III and from 2 August 2028 for AI in regulated products under Annex I.

For most agents in the back office, marketing or sales, these articles are not directly applicable in legal terms. But they describe very precisely what effective oversight looks like, which makes them a good benchmark for any approval gate. The legal framework is explained in detail in the guide [EU AI Act in Germany (in German)](https://andreasindorf.com/eu-ai-act-deutschland/).

## The underestimated danger: automation bias

An approval gate only protects if the human actually checks. That is exactly what becomes harder over time. The systematic review [Automation bias: a systematic review of frequency, effect mediators, and mitigators](https://academic.oup.com/jamia/article/19/1/121/732254) by Goddard, Roudsari and Wyatt defines automation bias as the tendency to over-rely on automation. According to the review, amplifying factors include trust in the system, high workload, task complexity and time pressure. As remedies, it names training, clear user accountability and design choices such as displaying confidence information and presenting information rather than direct recommendations.

For approval gates, this means:

- **Show the reasoning, not just the result:** The reviewer sees which sources and data the agent relies on, not just the finished draft.
- **Make uncertainty visible:** Cases without a suitable source or with contradictory data are flagged.
- **Plan review time:** Anyone expected to handle 200 approvals a day on the side stops checking.
- **Name the accountable person:** Every approval is assigned to one person, not to a group.
- **Seed control cases:** Occasionally inserting deliberately flawed test cases shows whether errors are still being caught.

## Metrics for approval gates

Whether a gate works is not a matter of gut feeling but of numbers from the log:

| Metric | What it shows | Warning sign |
|---|---|---|
| Correction rate | share of approvals with substantive changes | drops to zero while the error situation is unchanged |
| Rejection rate | share of rejected suggestions | persistently high: agent or level does not fit |
| Review time per approval | time between submission and decision | a few seconds for complex cases |
| Lead time to approval | waiting time at the gate | rising: bottleneck, missing deputy |
| Errors after approval | errors that reach the customer despite approval | every case points to automation bias |
| Control case detection | share of test errors caught | declines over time |

A persistently low correction rate can mean one of two things: the agent is good enough for a higher level, or the reviewers are rubber-stamping. Only the control cases and the errors after approval reveal which is true. How to measure knowledge AI with similar rigour is described in the article [Measuring RAG quality](https://andreasindorf.com/en/rag-evaluation/). A methodological framework for risk management as a whole is the voluntary [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework).

## From my own products

In the company cockpit I use to run my own business, one rule applies without exception: an approval gate is always held by a human. No agent approves another agent. The design decision behind this is simple: as soon as one model controls another, in the end nobody is accountable any more. Every task has one of four autonomy levels, a kill switch stops any specialist immediately, and the AI Act log records who approved or rejected what and when. The most important lesson from operations: the log is not just evidence, it is a management tool. Only the log tells you whether a task may move up a level or whether reviewing has become routine.

How to anchor this in an organisation with many agents is described in the [AI department guide](https://andreasindorf.com/en/ai-department-guide/); you will find my approach under [AI department](https://andreasindorf.com/en/services/ai-department/).

## Frequently asked questions

### What is an approval gate for AI agents?

An approval gate is a fixed point in the process at which the agent stops and a human reviews and decides. It belongs in front of every action that is hard to reverse, has an external effect, moves money or directly affects people.

### Does the EU AI Act require human in the loop?

For high-risk AI systems, Article 14 requires effective human oversight, including the ability to override outputs and to stop the system. Article 26 obliges deployers to assign oversight to competent people and to keep logs for at least six months. For most other agents this is not mandatory, but it is a good benchmark.

### How do I prevent approvals from becoming mere click-throughs?

Show reasoning and sources rather than just results, flag uncertain cases, plan realistic review time, assign each approval to one person and occasionally seed control cases with deliberate errors. The detection rate for these cases shows whether reviewing is still happening.

### When may an agent be given a higher autonomy level?

When the correction and error rates have been consistently low over a predefined period and the control cases show that reviewers are still catching errors. Promotion happens per task, not per agent, and if quality drops, the task is demoted immediately.

### Can an AI agent handle approval for another agent?

A second agent can pre-check and flag anomalies. The approval itself should stay with a human, because otherwise nobody is accountable any more and oversight within the meaning of Article 14 of the AI Act is not carried out by natural persons.