Human-in-the-Loop: Why Critical Decisions Must Stay With People
Short answer: Human-in-the-loop means that at defined points in an automated process, a person reviews, decides or stops the flow. For decisions that are legally binding for a person or significantly affect them, the GDPR sharply limits purely automated decision-making [1]. For high-risk AI, the AI Act requires effective human oversight, and after the Digital Omnibus those rules apply from 2 December 2027 [3][5]. So automate capturing, checking and preparing, and leave decisions with consequences to people.
The better AI gets at reading documents and making suggestions, the more tempting it is to take people out of the process altogether. In some places that is legally risky, and in everyday work it is a bad idea. This article shows what the GDPR and the AI Act say in 2026, which decisions should stay with a person and how to anchor the review in the process so that it amounts to more than a click.
1. What human-in-the-loop means in practice
A person in the process only protects you if they can change the outcome. For high-risk systems, Article 14 of the AI Act spells this out in surprising detail. Whoever takes on oversight should be able to [3]:
1. understand the system's capabilities and limits and monitor how it runs,
2. stay aware of the tendency to rely too much on its output (automation bias),
3. interpret the output correctly,
4. decide in a given situation not to use the system, or to disregard, override or reverse its output,
5. intervene or halt the system with a stop button or a similar procedure.
These five points work as a checklist even if your process is not high-risk.
2. What the GDPR requires
The European Commission sums up the rule like this: nobody should be subject to a decision based solely on automated processing that is legally binding or significantly affects them. Its example is the automatic refusal of an online credit application [1].
Such a decision is allowed in only three cases. It is necessary for a contract, a law permits it with safeguards, or the person has explicitly consented [1]. Even then, safeguards are required. The person must be told about the logic involved, the consequences and their right to human intervention, and must be able to contest the decision. That duty does not apply where the decision rests on a law [1].
On 7 December 2023, in the SCHUFA case (C-634/21), the Court of Justice read the term broadly. An automatically calculated score counts as an automated decision in its own right when banks give it a determining role in granting credit [2]. What counts is who actually decides. Our take: a person who merely nods through an automatically generated value is weak protection.
3. What the AI Act requires
Article 14 is aimed at providers. A high-risk system must be designed so that people can effectively oversee it while it is in use [3]. Article 26 is aimed at organizations that use such a system. They assign oversight to people with the necessary competence, training and authority, and they support them. They keep the logs the system generates for at least six months, as far as those logs are under their control. And they inform people about whom an Annex III high-risk system makes or prepares decisions [4].
The timeline has shifted. Under the Digital Omnibus on AI, the rules for high-risk systems under Annex III apply from 2 December 2027, and for high-risk AI in regulated products under Annex I from 2 August 2028 [5]. Article 6 and Annex III determine whether your use is high-risk. The Commission offers a compliance checker for this [5].
4. Which decisions should stay with a person
Three questions help draw the line. Does the decision have legal or similarly significant consequences for a person [1]? Is a mistake hard to undo later, like a payment that has already gone out? And is the case outside every rule because it deviates from anything the system knows? If the answer to any of these is yes, a person should make the decision. That is our recommendation, not statute.
| Process | Runs automatically | A person decides |
|---|
| Invoice approval | Capture, reading by OCR and AI, matching against order and goods receipt | Approval by the assigned approvers, especially when something deviates |
| Payment run | Collect due payments and check them against approvals | Finance approves the run |
| Contract approvals | Upload, route to reviewers, track deadlines | Legal, purchasing and management sign off |
| Applications | Structure the résumé with OCR and AI, extract qualifications | Selection, invitation and rejection |
| Four-eyes review | Route critical transactions to two independent reviewers | Both reviewers must approve |
5. How to make the human review effective
Someone who only sees the suggestion "approve" will mostly approve. That is automation bias, which Article 14 names explicitly [3]. So the reviewer should see the original document, the AI result and the deviations side by side.
The review needs real options. If all a person can do is confirm, they are not reviewing. Rejecting, returning and changing must be possible, and a running case must be stoppable [3].
The reviewer needs authority and a deputy. For high-risk systems, Article 26 requires competence, training and authority [4]. In practice, add a deadline after which the case escalates to the deputy or the line manager, so that no decision sits in an inbox.
And every decision must be traceable, with person, timestamp and outcome. For high-risk systems, logs have to be kept for at least six months [4].
6. How FireStart implements human-in-the-loop
In FireStart, you define in the BPMN 2.0 process model where a person decides. Before that point, AI does the groundwork, such as reading and classifying documents and preparing a decision. Approvals and critical decisions stay with people. FireStart logs every step with person, timestamp and decision. You choose whether the platform runs in the cloud, hosted in Germany, or on-premises.
Software does not replace legal assessment. Whether your use of AI counts as high-risk, and which duties follow from that, is something to settle with your legal counsel.
7. Five questions for your process
1. Does the decision affect a person legally or significantly [1]?
2. Can the person actually change the outcome or stop the case [3]?
3. Do they see enough to decide, and not just the AI's result?
4. Who is responsible, who stands in, and what happens when the deadline passes?
5. Where is every decision recorded with person and timestamp?
Frequently asked questions
What does human-in-the-loop mean?
Human-in-the-loop means that at defined points in an automated process, a person reviews, decides or stops the flow. The AI prepares, and the person carries the decision with consequences.
Is human-in-the-loop required by law?
In certain cases, yes. The GDPR sharply limits decisions based solely on automated processing that legally or significantly affect a person [1]. The AI Act requires human oversight for high-risk systems [3]. For all other processes it is a recommendation.
Is it enough if a person clicks "Approve" at the end?
That depends on whether the person can actually influence the decision. In 2023 the Court of Justice held that an automatically generated value counts as a decision in itself when the deciding party gives it a determining role [2]. Merely nodding it through is therefore weak protection.
When do the oversight duties of the AI Act apply?
For high-risk systems under Annex III from 2 December 2027, and for high-risk AI in regulated products under Annex I from 2 August 2028. That follows from the Commission's timeline after the Digital Omnibus on AI [5].
Which decisions should I not leave to AI alone?
All decisions that are legally binding for a person or significantly affect them [1], plus decisions whose errors are hard to undo. That is our recommendation, based on the GDPR rule.
Does human-in-the-loop slow the process down?
Only at the points where a person is meant to decide. Capturing, checking and assigning run automatically beforehand, and the reviewer receives a prepared decision.
Sources
1. European Commission: Are there restrictions on the use of automated decision-making?, https://commission.europa.eu/law/law-topic/data-protection/rules-business-and-organisations/dealing-citizens/are-there-restrictions-use-automated-decision-making_en
2. Court of Justice of the European Union: Press release No 186/23, judgments in Case C-634/21, SCHUFA Holding (Scoring) (7 December 2023), https://curia.europa.eu/jcms/jcms/p1_4213242/en
3. European Commission, AI Act Service Desk: Article 14, Human oversight, https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-14
4. European Commission, AI Act Service Desk: Article 26, Obligations of deployers of high-risk AI systems, https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-26
5. European Commission, AI Act Service Desk: Timeline for the Implementation of the EU AI Act, https://ai-act-service-desk.ec.europa.eu/en/ai-act/timeline/timeline-implementation-eu-ai-act
FireStart wird in der EU gehostet (DSGVO-konform, EU-Datenspeicherung). Website: www.firestart.com. Kontakt: sales@firestart.com.
