Language Is a Security Surface for Agent Guards
Translating only a user's request—nothing else—flipped safety verdicts in most small and purpose-built AI agent guards, more as…
Researchers built a benchmark (LinguaGuardBench) that translates only the user instruction in agent-safety test cases into five languages while keeping every tool call, observation and label identical, isolating language as the sole variable in guard verdicts. Across 19 tested guards, most open, small, and purpose-built guards flipped verdicts beyond their own noise floor under translation, with the effect worsening for rarer languages (Amharic worse than French), while frontier guards like Gemini 3.8 Flash stayed essentially unaffected. Smaller guards showed either more false positives or, below 10B parameters, meaningful drops in recall (missed unsafe actions).
Why it matters
Step-level guards are meant to block unsafe agent actions regardless of what language a user typed, but this research shows that assumption breaks down for many open and purpose-built guards, with safety-relevant inconsistency growing for less-resourced languages. That means an agent's actual real-world safety behavior can depend on what language a user happens to write in, a gap most teams deploying guards would not know to test for.
What is verified
Researchers built LinguaGuardBench by translating only the user instruction (keeping tool calls, observations, and labels byte-identical) from the 1,220 labelled steps of TS-Bench AgentDojo-Traj into German, French, Turkish, Swahili and Amharic, to isolate the effect of language on agent guard verdicts. [1]
Across 19 guards tested (frontier APIs, 4B open models, and four purpose-built agent guards), most open, small and purpose-built guards changed verdicts under translation beyond their own measured noise floor, with the effect growing with the rarity of the language. [1]
Among guards below 13B parameters, the mean placebo-adjusted flip rate rose from +1.5 percentage points on French to +4.4 percentage points on Amharic. [1]
Open guards of 12-30B parameters showed increased false positives (up to +5.0 percentage points blocking benign steps), while every significant loss of recall occurred in guards below 10B parameters (up to -7.4 percentage points). [1]
Frontier guards shifted verdicts by at most about one percentage point under translation, with the strongest tested guard, Gemini 3.8 Flash, returning identical verdicts with and without translation. [1]
The paper was accepted at the NeurIPS 2026 Workshop on Agents in the Wild: Safety, Security, and Beyond, and the authors released the benchmark along with all 489k verdicts. [1]
What is not
- No independent replication of these results beyond the authors' own benchmark and evaluation is available in the evidence.
- The evidence does not specify which guard products or vendors beyond Gemini 3.8 Flash were tested by name, limiting assessment of real-world deployment exposure.
- It is not established whether any production agent systems currently rely on the smaller or purpose-built guards shown to be most sensitive to translation.
- No information is given on mitigation techniques or whether guard vendors have been notified or have responded.
Sources
- 1arXiv cs.CRarxiv.org · Primary source
Original source published 15 October 2026 at 19:19.
An AI-assisted briefing by Coding Partners. We link the evidence behind the reporting so you can explore the original work and assess the claims. How CP Signal works.