Your "Autonomous Agent" Is One Confident Hallucination Away From a Lawsuit. Stop Granting Default Autonomy.

I have stopped counting the demos I watched this year where an "autonomous AI agent" was handed write access to a production database, an inbox, or a Stripe key and told to "handle it." Every one impressed in the script and terrified me in the implications. The autonomy dial in 2026 is stuck at 10. Almost nothing in production needs it above 3. The bill is starting to come due.

Why It Matters Now

Every major lab shipped a fully-autonomous agent product in the last twelve months — Operator, Computer Use, Project Mariner, Manus, Devin, Agentforce. The shape is identical: the model picks the action, the model calls the tool, no human approves. The pitch is "let your agent handle it." The implicit promise is that approval gates are friction, not safety.

That promise is wrong at scale. Klarna's customer-service agent was the most expensive public retraction in agent history — eighteen months unsupervised, then a public reversal and a re-hire of human agents. Anthropic's Responsible Scaling Policy flags "misaligned autonomous actions" as the failure mode that gates higher capability tiers. The people pretending otherwise are trading a future incident for a current sales cycle.

The Argument

The right default is human-in-the-loop with explicit, narrow approval for every irreversible action. Not fully autonomous. Not human-on-the-loop. The agent proposes; a human — or a deterministic guardrail — approves anything that costs money, sends a message, deletes data, mutates a record, or leaves a trail a lawyer can subpoena. The model is the fastest junior employee you have ever hired, and the correct policy for your fastest junior employee is "ask before anything irreversible." That is not a concession to LLM weakness. It is a concession to the fact that LLMs are confidently wrong in ways humans are not.

Three reasons.

Hallucination rate does not drop to zero at the frontier. Frontier models still produce confident factual errors at 0.5% to 4% on production traffic. At a 1% per-call error rate, an agent making ten tool calls per task gets something materially wrong on roughly 10% of runs. That is not a rounding error.

Agents do not have calibrated uncertainty. A human who is unsure pauses. The frontier models do not. They commit. They confidently click "confirm" on a destructive dialog when they have no real basis for the choice. This is the failure mode behind the Operator and Computer Use screenshots circulating all year — the agent paid the wrong invoice, sent the wrong email, filed the wrong ticket, and the user did not find out until the customer did. The model had no internal signal to wait.

Audit and accountability collapse. When the agent makes the call end-to-end, you have a log, an outcome, and no one to ask "why." The model cannot testify. The answer to "who decided this" is "the model did." That is not an answer any regulator, customer, or insurance carrier in 2026 accepts.

What I Am Not Saying

I am not arguing agents should not act. Long-running workflows — security triage, infrastructure remediation, scheduled research — genuinely benefit from the pattern, with the right guardrails. I am arguing "fully autonomous" should be earned per task, not granted by default. The teams shipping successfully run their agents with explicit approval gates for irreversible actions and deny lists for anything touching money, identity, or production state. The teams without those gates are the ones whose postmortems I will be reading in Q4.

What I Want To See Change

The default autonomy level in every framework should ship at 2 out of 10, not 10. Every irreversible tool call should require an explicit human approval channel by default, with the user opting into "skip approval" only after observing the agent get it right twenty times in a row. Right now the unsafe path is one toggle; the safe path is three abstractions of middleware.

Stop buying "fully autonomous." Start buying "autonomous where it is safe, human-approved where it is not." The agents are good enough to propose. They are not good enough to commit. The labs that price the difference honestly will own the next decade. The ones that pretend the dial goes to 11 will own the next postmortem.

— Mr. Technology