Who Drives the Loop?

Autonomous, human-in-the-loop, and AI-in-the-loop get talked about as levels of safety. They are better read as three answers to one question: who drives at this point in the work?

The short version

"There's a human in the loop" is usually said to end a conversation about risk. It shouldn't. Putting a person somewhere in an automated process does not make the process safe. That depends on where they sit, what they can see, how fast the machine is moving, and whether they can actually stop it.

There are three common setups. In an autonomous loop the machine drives the whole cycle. In a human-in-the-loop setup the machine drives but pauses for a person to rule before it continues. In an AI-in-the-loop setup the person drives and the machine proposes alongside them.

None of these is the grown-up version of the others. Each fits some work and fails badly on other work, and most real workflows use all three at different points.

What a loop is

A loop is any process that looks at a situation, decides, acts, and then uses what happened to adjust the next pass. US Air Force colonel John Boyd's OODA loop (observe, orient, decide, act) is the best-known version. When people draw it for AI systems it usually has four phases instead, because deciding and acting get merged and feedback gets a phase of its own.

1. Input (observe). The system takes in the situation. For an agent, that means reading its context, polling an API, or ingesting data.
2. Processing (orient). It makes sense of what came in against what it already knows. This is where an agent reasons.
3. Execution (decide and act). It commits and changes something: a tool call, a database write, a message sent.
4. Feedback. It measures what happened, and the result feeds into the next pass's input.

Nothing in that cycle says who does each phase. The three setups below are three answers to that, and the question worth asking is who holds the wheel at each point where the loop's output lands somewhere that matters — a customer's inbox, a database, a doctor's worklist, production.

The autonomous loop

Here the machine senses, reasons, acts, and checks its own result, with no person involved turn by turn. It suits high-volume, low-stakes work whose actions can be undone: sorting, extraction, first-pass triage of things a human will see later anyway.

Its characteristic failure is compounding error. In a chain of dependent steps the chances multiply. If each step is right 95% of the time and every step depends on the one before, ten steps come out right about 60% of the time, and twenty steps about 36%.

Read that table as a warning rather than a forecast. It assumes each step fails independently, and agents break that assumption in both directions. They get worse once their own earlier mistakes are sitting in their context, which researchers call self-conditioning. They also re-plan and recover, which the arithmetic ignores. The honest use of the table is to decide that a long chain needs checks between its steps, and then to measure your actual run.

Since nobody watches each turn, the oversight has to live inside the system. That means a structured check on each step's output before the next step uses it, a hard cap on steps, tool permissions scoped to the job, a budget per run, and a separate evaluator that decides when the work is done, so the agent isn't grading itself. It also needs something that notices when those checks get worse, because nobody else will.

Human-in-the-loop

The machine does the work and stops at a defined point for a person to approve, correct, or reject it before carrying on. This is the setup for consequential decisions where machine speed helps but a person has to own the call: medical triage, content moderation, payments above a threshold.

It fails when the gate turns into a rubber stamp. People tend to defer to a confident system, especially under time pressure. Show a recommendation with a big Approve button underneath, and the review becomes a formality. The machine is effectively deciding and the person is signing.

That costs more than bad decisions. The sociologist Madeleine Clare Elish called it the moral crumple zone. When control is spread across a complex system but responsibility is not, blame lands on the human nearest the failure, the way a car's crumple zone absorbs a crash to protect the cabin. One of her examples is Three Mile Island, where early accounts emphasised operator error even though a clogged filter in the feedwater system, and a control room that misrepresented the plant's state, set the operators up to fail.

The pattern has reached AI. After an Uber test vehicle killed a woman in Tempe, Arizona in 2018, the safety driver was charged with negligent homicide; in 2023 she pleaded guilty to endangerment. Her job had been to supervise a system making decisions faster than she could follow.

A gate that works needs friction that makes the reviewer think. Put the evidence and the reasoning on screen next to the output, and ask for more than a click: choosing the finding that supports the call, say, or picking hold, redirect, narrow, or reject. Then watch one number closely, which is how often the reviewer says no. If a gate hasn't overridden the machine in months, that tells you very little about the machine and quite a lot about the gate.

Qure.ai's qER is an example that respects the line. The FDA cleared it in 2020 as a Class II triage device (K200921). It reads non-contrast head CT scans for four findings — intracranial haemorrhage, mass effect, midline shift, and cranial fracture — flags suspected cases, and notifies clinicians. In the study behind that clearance, qER sent its notification in 2.11 minutes on average, while a scan waited 65.54 minutes on average to be opened under the standard of care.

What makes it a well-placed gate is what it leaves alone. Its labelling says it is not a diagnostic device, it does not remove cases from the reading queue, and notified clinicians stay responsible for viewing the full images according to the standard of care. The machine reorders the work, and the person still makes the call.

AI-in-the-loop

Now the person drives. The machine works alongside, suggesting, checking, and drafting, but the decision and the direction stay with the human. This fits work where the hard part is framing the problem: strategy, design, writing, most software engineering.

The risks are quieter. A tool that keeps interrupting with suggestions nobody asked for breaks concentration and makes the person spend attention turning things down. Over a longer stretch, a skill the machine always assists is a skill that stops getting practised.

What helps is showing each suggestion as a difference against the person's own work, so they can see exactly what would change before accepting it. Code editors like Cursor do this with a diff view of proposed edits. The suggestion sits on top of the work without replacing it, and nothing arrives that the person didn't ask for.

How to choose, point by point

Pick a setup for each point where the loop's output lands, rather than one for the whole system. Four questions do most of the work.

1. How long is the chain, and how reliable is each step? A long run of dependent steps with nothing checking between them shouldn't run unattended.
2. Can the action be undone? If money moves, a message goes out, or a record is deleted, the step isn't autonomous, however accurate it has been so far.
3. Can the reviewer keep up? If the machine produces faster than a person can genuinely check each output, a human-in-the-loop gate becomes a stamp. Slow the loop, sample by type of work, or move the person into the driver's seat, where they make the decision instead of auditing someone else's.
4. Does the person who answers for it have the controls? Whoever is accountable for a step must be able to stop, override, or reverse it at the speed the loop runs, with the evidence in front of them. The EU AI Act's human-oversight article asks for much the same: people able to disregard, override, or reverse the output, and to stop the system. Someone held accountable without those controls is in the crumple zone, whatever their job title says.

These choices should move over time, and only on evidence. A step earns more autonomy from a record of how it has performed on that kind of work, and loses it after a miss. Granting autonomy because the tool has felt reliable lately is how over-trust gets built in.

The loop above the loop

All three setups correct mistakes in the output. By default, none of them questions the setup itself.

Chris Argyris called this the difference between single-loop and double-loop learning. A thermostat that turns the heat on when the room drops below 68 degrees is single-loop; double-loop learning asks whether 68 is the right number. For an AI workflow, ask what would make anyone change the loop's goals, its checks, or who drives where. If nothing would, the workflow can only get better at doing what it already does.

This is where the Two-Speed Engine puts it: a human loop that forms judgment, a machine loop that executes, and a designed boundary between them. The three setups above are the three ways to set any single point on that boundary. The playbook's machine-loop section and its loop-design skill walk through choosing them for a real workflow.