Do AI Coding Assistants Check Before They Install? A Pre-Registered Demand-Side Audit of Trust Signals in the Research Software Supply Chain
In 1,920 controlled trials, AI coding assistants opened a package's cryptographic trust signal before installing it just 9 times.
A pre-registered audit tested whether AI coding assistants check SBOMs, signed releases, build attestations, or official-channel declarations before installing software from six research-software projects. Across 1,920 registered trials (three models, two operating modes, plus a supplement on three frontier models), assistants opened any provenance signal in only 0.5% of trials and never ran a verification command, regardless of whether signals were present, missing, mismatched, or forged with a wrong issuer.
Why it matters
This is one of the first measured (not surveyed) demonstrations that publishing supply-chain trust signals does nothing on its own if the software agent consuming them never reads them. It reframes a widely assumed mitigation for agentic-AI package attacks (malicious packages, typosquatting, compromised maintainer accounts) as currently ineffective in practice, because verification isn't happening at the assistant level regardless of model capability or cost.
What is verified
A pre-registered controlled study ran 1,920 registered trials across six open-source research software projects (three HPC, three quantum computing) using three models under two operating modes, plus a supplement on three frontier models. [1]
The assistant opened any provenance signal (SBOM, signed release, build attestation, declared official channel) before installing in 9 of 1,920 registered trials (0.5%), and 0 of 384 control trials. [1]
No trial ran a verification command, so the presence of trust signals had no measurable effect on behavior. [1]
The model that verified most often cost $0.10 per trial; the most capable model tested, at $1.00 per trial, verified nothing. [1]
The study included nine modified copies per project: no signal, one per signal class, wrong-issuer signature/attestation variants, all four signals combined, and one with documented metadata conflicts. [1]
The authors deposited protocol, seed, panel, and analysis plan with a DOI before running any trial, and released the per-trial cost ledger, protocol, and logs. [1]
What is not
- The specific three models and three frontier models tested are not named in the abstract.
- Results are limited to six research-software projects (HPC and quantum computing); generalization to mainstream package ecosystems (npm, PyPI, crates.io) used in typical client work is not established.
- The exact tools/harnesses used to 'operate an assistant' (e.g., which coding agent products) are not specified.
- No detail on what 'with and without an approval step' concretely means in terms of user interaction or permissions model.
The case for
"
The case against
The study covers only six research-software projects in HPC and quantum computing, not mainstream package ecosystems (npm, PyPI) that a typical agency uses daily, and the specific assistant products/harnesses tested aren't named, so it's unclear how directly this generalizes to tools like Claude Code, Codex or Cursor as configured in real client workflows.
What to do with this
Treat this as a strong signal to add explicit, tool-enforced dependency verification into any pipeline where an AI agent installs packages, rather than trusting model judgment or the mere presence of SBOMs/signatures to matter — while noting the direct evidence is from research software, not the npm/PyPI ecosystems used in most client work.
Sources
- 1arXiv cs.SEarxiv.org · Primary source
Original source published 7 September 2026 at 18:55.
An AI-assisted briefing by Coding Partners. We link the evidence behind the reporting so you can explore the original work and assess the claims. How CP Signal works.