The Approve Button Is Not a Control

Almost every organization that has approved AI in the last eighteen months approved it with a condition attached. The condition is nearly always the same sentence, and it usually appears in the policy document within the first page:

A human reviews each action before it is taken.

It is a reasonable-sounding sentence. It has cleared legal review at hospitals, county agencies, universities, and accounting firms. It is the reason a great many AI pilots were allowed to exist at all.

Last week, someone put a number on it.

The number

On August 5, Alex Wauters published the results of a browser-based simulation on the Scale X blog. The setup is simple: you play the human in the loop for an AI coding agent. Commands come at you under time pressure. Some are routine. Some mean the agent has been compromised and is shipping your credentials to a stranger. You approve or you deny.

Across more than 40,000 runs and 409,000 individual decisions, the average person caught 66.3% of the threats. One in three got through.

The post reached the front page of Hacker News, where it drew 338 points and 245 comments — an unusually heavy comment-to-point ratio, which is generally the signature of a finding that practitioners recognize rather than merely admire.

The rest of the distribution is worse than the average suggests. 32.9% of sessions ended with a negative score — the damage approved outweighed everything caught. 35.2% of players caught every single threat, but only 20.8% managed it without also blocking a fifth or more of the safe commands. And 7% approved everything put in front of them.

The author is candid about the limits: roughly 34% of the commands in the game were threats, a density no real workday matches, and the clock added artificial pressure. Both cut in the direction of making players more alert than they would be at 4:30 on a Thursday. Treat the headline figure as a comparison of threat types, not a forecast of your own hit rate.

That comparison is the part worth reading twice.

People catch the wrong things

The miss rate was not spread evenly. It sorted almost perfectly by how alarming the command looked:

What it was | How often it got through

Obviously destructive (rm -rf /) | 11.7%

Persistent changes (scheduled-job injection, config hijack) | 23.8%

Exfiltration and code execution | 33.4%

Scope violations (reading cloud credential files) | 35.0%

The commands that announce themselves get stopped. The commands that quietly walk out with your credentials get through three times as often.

The clearest case in the data is a command called npm run analyze — a routine developer instruction to inspect a project's file sizes. It was approved 64.7% of the time, the single most-missed item in the study. In the simulation, that innocuous name was wired to send the output to an unknown external server, and the game showed the player that wiring in the log directly above the prompt. Two thirds approved anyway.

Pool the three commands built on that pattern and they were missed 52.5% of the time, against 28.4% for every other exfiltration-style attack in the study. Putting a hostile payload behind a familiar name roughly doubled its success rate — even with the evidence on screen.

There is a mirror-image failure in the same dataset. A perfectly legitimate command pointing the developer's package manager at the company's own internal server was blocked 59% of the time. The reviewers who were vigilant enough to catch the attacks were often the same ones stopping ordinary work. The study's name for that outcome is the "Human Bottleneck," and if you have ever watched an AI pilot die of friction, you have seen it.

Why this is a governance finding, not a developer story

It is tempting to file this under engineering. It is not an engineering finding. It is a finding about a control that thousands of organizations have written into policy.

Approval is a control that assumes attention is free and constant. It is neither. The study quotes Anthropic's own observation on the subject: "The more approvals a user sees, the less attention they pay to each, becoming over time much less diligent." The data shows the shape of that decay inside a game that lasts a couple of minutes. Your staff are on month eleven.

There is a second problem, raised in the Hacker News thread and harder to solve than fatigue: at the moment of approval, a great many commands are genuinely ambiguous. The same instruction is safe or catastrophic depending on a file the agent edited four steps earlier. Asking a person to approve it is asking them to certify something the prompt does not contain. Slower reading does not fix that. Nothing at that layer does.

So the honest reading is not "people are careless." It is that per-action human approval is a weak control being asked to carry an organization's entire AI risk position.

What actually holds

The fix is not to remove the human. It is to stop spending the human on the wrong decision. Three shifts do most of the work:

Constrain the environment, not the click. An agent that has no route to your credential store and no path to an unapproved external destination does not need to be caught asking. Decide once, at the boundary, when you are calm — instead of forty times a day, at speed. Every approval you can delete is one you cannot fatigue on.

Reserve human judgment for consequences, not commands. People are good at "should this money move, should this go to the client, should this touch a patient record." They are demonstrably poor at auditing routine-looking instructions in volume. Put the approval where the consequence is, and the click means something again.

Log everything, and read the log on a schedule. Real-time review is where attention fails. Retrospective review is where it works — nobody is under a clock, and patterns are visible that single decisions never are.

None of this is exotic. It is the same principle every regulated institution already applies to money and to medical records: you do not protect the account by asking the teller to concentrate harder. You limit what the teller can do, and you reconcile afterward.

The question to ask this week

If your organization is running AI agents, or is about to, find the sentence in your policy that says a person reviews each action. Then ask the only question that matters about it: what would have to be true for that to work?

Constant attention, at volume, under deadline, with full context at the moment of the click. The data says that is not the environment your people are working in.

The approve button is not a control. It is a record of who was watching. Build the control somewhere else.

Sources: Alex Wauters, "Humans missed 1 in 3 threats approving AI agent commands across 40,000 plays," Scale X, August 5, 2026 — https://scalex.dev/blog/ai-agent-permissions-stats/ · Hacker News discussion (338 points, 245 comments as of August 10, 2026) — https://news.ycombinator.com/item?id=49195468