Approve What, Exactly? Effect Binding and Trusted Rendering for Agent Step-Up Authorization
Testing ten agent runtimes found none kept a tamper-proof approval record, and four let post-approval arguments change silently.
A research paper defines twelve concrete attacks against the "human approval" step that agentic AI systems use before sending money, deleting files or emailing outsiders, then tests them against AP2, Microsoft's action-bound approval design, and MCP elicitation. Auditing ten agent runtimes, the authors found none produced a third-party verifiable approval record, none enforced measurable flood limits, and four of eight runtimes with measurable binding allowed the underlying action's arguments to change after approval. Their reference fix, a gateway called ASU that renders approval cards from a canonical effect descriptor and binds them to a phone passkey signature, cut forged-narration attacks from 0.54 to 0.03 success rate and raised LLM-approver attack catch rate from 0.30 to 0.98 in the
Why it matters
As agentic AI systems gain the ability to take consequential actions (payments, deletions, external communication), the "human-in-the-loop approval" step is often assumed to be a reliable safety check. This research demonstrates that current implementations across major protocols and ten real agent runtimes can be subverted so the user approves something other than what actually executes, undermining a safety mechanism many teams treat as sufficient.
What is verified
The authors define twelve attacks on agent approval paths (e.g. decoy recipients, amount cancellation, card padding, label spoofing, enrollment squatting, threshold splitting) and test them against AP2, Microsoft's action-bound approval design, and MCP elicitation. [1]
An audit of ten agent runtimes found no third-party verifiable approval record, no measurable flood limits, and post-approval argument changes in four of eight runtimes with measurable binding. [1]
The authors built a reference defense called ASU (Agent Step-Up), a gateway that renders approval cards from a canonical effect descriptor and has a phone passkey sign the digest before executing only the stored call. [1]
Tamarin formal verification proved ASU's binding, single-use and key-level non-repudiation properties under three stated hypotheses using 53 lemmas. [1]
Across 27,108 scripted trials and 609 AgentDojo attack pairs, the trusted card reduced forged-narration attack success from 0.54 to 0.03. [1]
With 15 language models acting as automated approvers, the trusted card raised attack catch rate from 0.30 to 0.98, while refusing 16 percent of harmless near-misses, mostly attributed to a placeholder artifact in the stimuli. [1]
A second device added nothing measurable over trusted on-host confirmation in these experiments, flooding still succeeded 0.45 to 0.60 of the time against a modeled approver, and approval cannot stop effects that policy already allows. [1]
What is not
- No independent replication or peer review of these findings beyond the arXiv preprint is cited.
- The paper does not specify which commercial agent products correspond to the ten audited runtimes.
- Real-world deployment data on how widely AP2, Microsoft's action-bound design, or MCP elicitation are used in production is not given.
- It is unclear whether the 16 percent false-refusal rate on harmless near-misses would be acceptable in production UX given it is partly attributed to a stimuli artifact.
- No information on timeline for ASU or similar defenses to be adopted by agent platform vendors.
Sources
- 1arXiv cs.CRarxiv.org · Primary source
Original source published 15 October 2026 at 03:11.
An AI-assisted briefing by Coding Partners. We link the evidence behind the reporting so you can explore the original work and assess the claims. How CP Signal works.